I'm trying to download a link within a table after navigating to a page using Python3 + selenium. After using selenium to click through links, and inspecting the elements on the latest-loaded page, I can see that my element is within a frame called "contents". When I try to access this frame, however, upon calling:
DRIVER.switch_to_frame("contents")
I get the following error:
selenium.common.exceptions.NoSuchFrameException: Message: contents
To gain more context I applied an experiment. Both the page that I loaded using DRIVER.get(URL) and the one to which I navigated had a frame called "menu".
I could call DRIVER.switch_to_frame("menu") on the first page, but not on the second.
DRIVER = webdriver.Chrome(CHROME_DRIVER)
DRIVER.get(SITE_URL)
DRIVER.switch_to_frame("contents") # This works
target_element = DRIVER.find_element_by_name(LINK)
target_element.click()
time.sleep(5)
DRIVER.switch_to_frame("menu")
target_element = DRIVER.find_element_by_name(LINK2)
target_element.click()
target_element = DRIVER.find_element_by_name(LINK3)
target_element.click()
DRIVER.switch_to_frame("contents") # Code breaks here.
target_element = DRIVER.find_element_by_xpath(REF)
target_element.click()
print("Program complete.")
I expect the code to find the xpath reference for the link in the "contents" frame. Instead, when attempt to switch to the "contents" frame, python run-time errors and cannot find "contents".
selenium.common.exceptions.NoSuchFrameException: Message: contents
this is because you are staying in child level of iframe which is 'menu' so inside that it can't able to find the iframe 'contents'.
First Switch back to the parent frame which is "contents", by using
DRIVER.switch_to.default_content()
and then try to go to the 'contents' iframe and perform actions, Now it should work.
Since contents appears to be a top level frame try going back to the top before selecting the frame:
DRIVER.switch_to.default_content()
DRIVER.switch_to.frame("contents")
As a thumb rule whenever switching frames you need to induce WebDriverWait for frame_to_be_available_and_switch_to_it() and you need to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use the following solution:
target_element.click()
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"frame_xpath")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "REF"))).click()
You can find a detailed discussion on frame switching in How can I select a html element no matter what frame it is in in selenium?
Here you can find a relevant discussion on Ways to deal with #document under iframe
Related
I am using headless Firefox on Selenium and XPath Helper to identify insanely long paths to elements.
When the page initially loads, I can use XPath Helper to find the xpath of any element of interest, and selenium can find the element when given the xpath.
However, several buttons that I need to interact with on the page open menus when pressed that are either small or take up the whole "screen". No matter their size, these containers are overlaid on the original page, and although I can find their xpaths using XPath Helper, when I try to use those xpaths to find the elements using selenium, they can't be found.
I've checked, and there's no iframe funny business happening. I'm a bit stumped as to what could be happening. My guess is that the page's source code is being dynamically changed after I press the buttons that open the menu containers and when I call find_element_by_xpath on new elements in the containers, the original source is being searched, instead of the new source. Could that be it?
Any other ideas?
As a workaround, I can get around this issue by sending keystrokes to the body of the page, but I feel this solution is rather brittle and likely to fail. Would be a much more robust solution to actually specify all elements.
EDIT:
With selenium I can find the export button, but not the menu it opens.
Here is the code for the export button itself:
The element of interest for me is "Customize Export" which I have not been able to find using selenium. Here is the code for this element:
Notice the very top line of this last image (cdk-overlay-container)
Now, when I refresh the page and do NOT click the export button, the cdk-overlay-container section of the code is empty:
This suggests my that my hypothesis is correct -- that when the page loads initially, the "Customize Export" button is nowhere in the source code, but appears only after "Export" is clicked, and that selenium is using the original source code only --not the dynamically generated code that appears after clicking "Export" -- to find elements
Selenium could find the dynamic content after doing
driver.execute_script("return document.body.innerHTML")
The WebDriverWait is what you need to use to wait for a certain condition of elements. Here is an example of waiting for the elements to be clickable before the click with a timeout in 5 seconds:
wait = WebDriverWait(driver, 5)
button = wait.until(EC.element_to_be_clickable((By.XPATH, 'button xpath')))
button.click()
wait.until(EC.element_to_be_clickable((By.XPATH, 'menu xpath'))).click()
identify insanely long paths
is an anti pattern. You can try to not use XPath Helper and find xpath or selector yourself.
Update:
wait = WebDriverWait(driver, 10)
export_buttons = wait.until(EC. presence_of_all_elements_located((By.XPATH, '//button[contains(#class, "mat-menu-trigger") and contains(.="Export")]')))
print("Export button count: ", len(export_buttons))
export_button = wait.until(EC.element_to_be_clickable((By.XPATH, '//button[contains(#class, "mat-menu-trigger") and contains(.="Export")]')))
export_button.click()
cus_export_buttons = wait.until(EC. presence_of_all_elements_located((By.XPATH, '//button[contains(#class, "mat-menu-item") and contains(.="Customize Export")]')))
print("Customize Export button count: ", len(cus_export_buttons))
Here's a simple example:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.dcor.state.ga.us/GDC/Offender/Query")
button = driver.find_element_by_id('submit2')
The last command fails no matter how long I wait (unlike previous questions and answers-a long time after the page loads), but when I'm going to the browser itself(the instance that Selenium created), the green "I agree" button has the id submit2(right click->Inspect element)...
What causes this behavior?
How can I make it work?
That's because form located inside an iframe, so you need to switch to that frame before locating element:
driver.switch_to.frame('iframe-content')
The desired element is within an <iframe> so you need to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable and you can use the following solution:
Code Block:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"iframe-content")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.oq-sub.btn.btn-success#submit2"))).click()
i have written parameters in webpage & applying them by selecting apply button.
1. apply button is getting clicked for web elements outside frame using below command
browser1.find_element_by_name("action").click()
apply button not getting clicked when saving the paramters inside a frame of web page using same command
browser1.find_element_by_name("action").click()
you need to switch to iframe
fist you need to find the iframe then switch to it then click
driver.switch_to_frame(driver.find_element_by_tag_name("iframe"))
or you could use xpath to locate the element
driver.find_element_by_xpath("//iframe[]")
I do not know the Python syntax, but in Selenium you have to switch to the frame much like you would switch to a new tab or window before performing your click.
The syntax in Java would be: driver.switchTo().frame();
As mentioned by the others:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[]"))) #elements are in a iframe, have to select it first
If there is a chance that it doesn't show up immediately you might want to build in this wait function.
Tells python to wait for max 10 secs, until frame is available and switches to it right away. I use Xpath to track down the iframe
Dont know how new you are, so provided the imports below:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Credit for this solution goes to #Andersson
Finding xpaths on pages running script
Upvote him there.
Couldn't find the duplicate report button. Title of this post makes it easier to find then the above mentioned question. (full disclosure, the link links to one of my own questions)
I am trying to scrape content of a page.
Let's say this is the page:
http://finance.yahoo.com/quote/AAPL/key-statistics?p=AAPL
I know I need to use Selenium to get the data I want.
I found this example from Stackoverflow that shows how to do it:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("http://finance.yahoo.com/quote/AAPL/profile?p=AAPL")
# wait for the Full Time Employees to be visible
wait = WebDriverWait(driver, 10)
employees = wait.until(EC.visibility_of_element_located((By.XPATH, "//span[. = 'Full Time Employees']/following-sibling::strong")))
print(employees.text)
driver.close()
My question is this:
In the above example to find Full Time Employees the code that has been used is:
employees = wait.until(EC.visibility_of_element_located((By.XPATH, "//span[. = 'Full Time Employees']/following-sibling::strong")))
How the author has found that s/he needs to use:
"//span[. = 'Full Time Employees']/following-sibling::strong"
To find the number of employees.
For my example page: http://finance.yahoo.com/quote/AAPL/key-statistics?p=AAPL how can I find for example Trailing P/E?
Can you please tell me the steps you took to find this? I do right click and choose Inspect, but then what shall I do?
A picture is worth of thousand words.
In web dev. tools (F12) you do the following steps:
Choose Elements tab
Press Element Selector button
With that button pressed you click an element in the main browser window.
In the DOM-elements window you right-click that highlighted element.
The context menu gets transpired and you choose Copy.
Choose Copy XPath in a sub menu. Now you have that element xpath in a console buffer.
NOTE!
The browser makes/composes an element xpath based on its own algorithm. It might not be the way you think or the way that fits to your code. So, you have to understand xpath in nature, be acquainted with it.
See what xpath the Chrome browser has issued for Trailing P/E:
//*[#id="main-0-Quote-Proxy"]/section/div[2]/section/div/section/div[2]/div[1]/div[1]/div/table/tbody/tr[3]/td[1]/span
'//h3[contains(., "Valuation Measures")]/following-sibling::div[1]//tr[3]'
Here I have the answer for all your confusions.
It will be better to look on some xpath tutorials and do practice from yourself, then you will be able to decide what you have to use .
There are so many site. You can start Here or Here
Now come to your Query -
Suppose I am using following xpath to locate the element
//h3/span[text()='Financial Highlights']/../preceding-sibling::div//tr[3]/td/span
Your requirement to find Trailing P/E in your page, definatly you will look unique xpath which won't change. If you try to find this using firepath it shows some lengthy xpath
Now you will check alternative and find another element (may be sibling, child or ancestor of your element) based on that you can to locate your element
in My case, first will find the Financial Highlights text which I will be able to find using //h3/span[text()='Financial Highlights']
Now I move its parent tag which is h3 and I will do this using /..
I have Trailing P/E element in just above the current node so move on just above node using /preceding-sibling::div
And finally find your element in that <div> like -//tr[3]/td/span
See the screens as well -
Step 1 :
Step 2 :
Step 3 :
Step 4 :
I am working with Selenium WebDriver and wrote a small code to find and element (i.e a button) and click on it.
Here is the HTML code for the button:
<input type="submit" name="j_id0:j_id2:j_id3:j_id4:j_id7" value="New Master Health Program" onclick="AddLink()" class="btn">
Here is the C# code for the Test Case:
IWebElement newMasterHealthProgramsLink = driver.FindElement(By.Name("j_id0:j_id2:j_id3:j_id4:j_id7"));
newMasterHealthProgramsLink.Click();
I tried using XPath as well:
IWebElement newMasterHealthProgramsLink = driver.FindElement(By.XPath("//input[#id='j_id0:j_id2:j_id3:j_id4:j_id5']"));
newMasterHealthProgramsLink.Click();
I found a solution saying that you must not have implemented Wait for this. Page does not wait to load completely and tries to find the element. So I added wait command but nothing useful happened. Still getting the same error:
TestAutomation.Driver.Login:
OpenQA.Selenium.NoSuchElementException : The element could not be found
Since your element is in an IFrame, you'll need to 'switch' to that IFrameusing the WebDriver API.
By default, Selenium will use the 'top' frame, and therefore any 'find element' queries will be directed to the most top frame, and ignore any child IFrames.
To solve this, 'switching' to the current IFrame directs Selenium to shove any requests to that frame instead.
driver.SwitchTo().Frame()
Note that you'll need a way of accessing the frame, either by it's ID, index (the top frame is 0, next frame down is 1, etc...) or name.
Another note is that any further requests will be directed to that IFrame, ignoring any others, which includes the top one...so if you need to access the top frame, you'll need to switch back:
driver.Switch().DefaultContent().