i have written parameters in webpage & applying them by selecting apply button.
1. apply button is getting clicked for web elements outside frame using below command
browser1.find_element_by_name("action").click()
apply button not getting clicked when saving the paramters inside a frame of web page using same command
browser1.find_element_by_name("action").click()
you need to switch to iframe
fist you need to find the iframe then switch to it then click
driver.switch_to_frame(driver.find_element_by_tag_name("iframe"))
or you could use xpath to locate the element
driver.find_element_by_xpath("//iframe[]")
I do not know the Python syntax, but in Selenium you have to switch to the frame much like you would switch to a new tab or window before performing your click.
The syntax in Java would be: driver.switchTo().frame();
As mentioned by the others:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[]"))) #elements are in a iframe, have to select it first
If there is a chance that it doesn't show up immediately you might want to build in this wait function.
Tells python to wait for max 10 secs, until frame is available and switches to it right away. I use Xpath to track down the iframe
Dont know how new you are, so provided the imports below:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Credit for this solution goes to #Andersson
Finding xpaths on pages running script
Upvote him there.
Couldn't find the duplicate report button. Title of this post makes it easier to find then the above mentioned question. (full disclosure, the link links to one of my own questions)
Related
Currently I'm trying to scrape something from a website. For that I need content of an email and so I use yopmail for that (https://yopmail.com).
In yopmail you have the mails on the left side on the screen with the mail subject under it. This text is the part I need.
[the mail view][1]
[the devtools code][2]
The problem now is that this code is not available in the page source. For what I red online it can be caused by javascript generation although, I'm not sure that is exactly the problem
I've tried multiple solutions:
attempt 1:
using beautifulSoup and locate the element (failed because not in the page source)
attempt 2:
tried locate element with xpath with the selenium driver (also unable to find)
attempt 3:
get the inner html of the body (still not available in that html)
driver.find_element_by_tag_name('body').get_attribute('innerHTML')
It feels like nothing works and also the other related posts here dont give me an answer that helps. Is there anyone who can help me with this?
[1]: https://i.stack.imgur.com/vTi0s.png
[2]: https://i.stack.imgur.com/nmBZ8.png
It seems like the element you are trying to get is inside an iframe, that's why you are not able to locate it. So first you have to switch to the iframe by using:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.ID ,'ifinbox')))
element = driver.find_element(By.XPATH, "//div[#class='lms']")
print(element.text)
When you are done you can switch back to default content by using
driver.switch_to.default_content()
NOTE: You need to import the following
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
newbie here. I've been reading the site for a while as I'm still new to coding but hoping you can help.
I've worked my way through some tutorials/worked examples on web scraping and am looking at the website http://enfa.co.uk/
I am trying to open an instance of Chrome using chromedriver with selenium in Python and click on one of the sidebar options on the website called 'Clubs' - located on the left of the homepage.
I've navigated to the element that needs to be clicked and taken the xpath to use in my code (simple use of 'inspect' in the Chrome dev tools when hovering over the 'Clubs' link, then copying the xpath). My code opens chrome fine (so no issues with Chromedriver and that part of the project) but I receive an error telling me the object has no click attribute.
I've tried returning the object and it states my list has no elements (which seems to be the problem) but am unsure why... am I using the incorrect xpath or do some websites react differently i.e. won't respond to a click request like this?
I have run my code on other sites to check I'm utilising the click function and it seems to work ok so I'm a little stumped by this one. Any help would be great!
Code:
chromedriver = "/Users/philthomas/Desktop/web/chromedriver"
driver = webdriver.Chrome(chromedriver)
driver.get("http:enfa.co.uk")
button = driver.find_elements_by_xpath("/html/body/table/tbody/tr[5]/td")
button.click()
Traceback (most recent call last):
File "sel.py", line 9, in
button.click()
AttributeError: 'list' object has no attribute 'click'
HTML of link I am trying to click
find_elements_by_xpath returns list of all web elements that match the criteria. You have to use to find_element_by_xpath instead of find_elements_by_xpath.
Also, iframe are present on your page so you need to switch to that frame before you perform any action on it. Kindly find below ssolution which is working fine for me.
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(executable_path=r"C:\New folder\chromedriver.exe")
driver.maximize_window()
driver.get("http:enfa.co.uk")
driver.switch_to.frame(2);
ClubsLink=WebDriverWait(driver, 20).until(
EC.presence_of_element_located((By.XPATH, "//span[contains(text(),'Clubs')]")))
ClubsLink.click()
Output after clicking Clubs link :
helpful link to locate element :https://selenium-python.readthedocs.io/locating-elements.html
Change button = driver.find_elements_by_xpath("/html/body/table/tbody/tr[5]/td") to button = driver.find_element_by_xpath("/html/body/table/tbody/tr[5]/td")
driver.find_elements_by_xpath returns a collection of element, not a single one, hence you cannot click it.
You can find some details and examples here.
I'm trying to download a link within a table after navigating to a page using Python3 + selenium. After using selenium to click through links, and inspecting the elements on the latest-loaded page, I can see that my element is within a frame called "contents". When I try to access this frame, however, upon calling:
DRIVER.switch_to_frame("contents")
I get the following error:
selenium.common.exceptions.NoSuchFrameException: Message: contents
To gain more context I applied an experiment. Both the page that I loaded using DRIVER.get(URL) and the one to which I navigated had a frame called "menu".
I could call DRIVER.switch_to_frame("menu") on the first page, but not on the second.
DRIVER = webdriver.Chrome(CHROME_DRIVER)
DRIVER.get(SITE_URL)
DRIVER.switch_to_frame("contents") # This works
target_element = DRIVER.find_element_by_name(LINK)
target_element.click()
time.sleep(5)
DRIVER.switch_to_frame("menu")
target_element = DRIVER.find_element_by_name(LINK2)
target_element.click()
target_element = DRIVER.find_element_by_name(LINK3)
target_element.click()
DRIVER.switch_to_frame("contents") # Code breaks here.
target_element = DRIVER.find_element_by_xpath(REF)
target_element.click()
print("Program complete.")
I expect the code to find the xpath reference for the link in the "contents" frame. Instead, when attempt to switch to the "contents" frame, python run-time errors and cannot find "contents".
selenium.common.exceptions.NoSuchFrameException: Message: contents
this is because you are staying in child level of iframe which is 'menu' so inside that it can't able to find the iframe 'contents'.
First Switch back to the parent frame which is "contents", by using
DRIVER.switch_to.default_content()
and then try to go to the 'contents' iframe and perform actions, Now it should work.
Since contents appears to be a top level frame try going back to the top before selecting the frame:
DRIVER.switch_to.default_content()
DRIVER.switch_to.frame("contents")
As a thumb rule whenever switching frames you need to induce WebDriverWait for frame_to_be_available_and_switch_to_it() and you need to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use the following solution:
target_element.click()
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"frame_xpath")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "REF"))).click()
You can find a detailed discussion on frame switching in How can I select a html element no matter what frame it is in in selenium?
Here you can find a relevant discussion on Ways to deal with #document under iframe
So I'm trying to identify this button to press it, but I get the exception:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element
The image below shows the button:
The weird thing, is that I already did identify the login button in the login page, but after I got to the following page, it's like identifying an element isn't possible anymore, could that be because the page doesn't allow for that?
#This one works
driver.find_element_by_class_name("urBtnStdNew").click()
#This one give the exception
driver.find_element_by_id("sapbi_snippet_TABSTRIP_CONTAINER_ITEM_1").click()
#This is the HTML of that element
<span class="urTbsTxtOff">DPD Trend</span></td>
Edit: Thank you guys, I didn't know about iframe's existence, so switching it solved my problem.
iframe = driver.find_element_by_xpath("//iframe[#name='iframe_Roundtrip_9223342']")
driver.switch_to.frame(iframe)
Use WebDriverWait and following xpath to click on the DPD Trend tab.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//td[#class='urTbsLabelOff urBorderBox']/span[#class='urTbsTxtOff'][contains(.,'DPD Trend')]"))).click()
Hamza,
WebDriverWait will not work. As you have stated you have tried using Thread.sleep() and WebDriver still cannot locate the element.
The issue looks to me like it is down to an incorrect element locator being used.
If the button in your screenshot is the one in the red box, and the code in the red box in the element inspector relates to it, then it looks to me like the ID should be TABSTRIP_CONTAINER_ITEM_1_tabStrip_tabstrip-itm-5.
So based on your example above the following code should locate the DPD Trend button and click it:
driver.find_element_by_id("TABSTRIP_CONTAINER_ITEM_1_tabStrip_tabstrip-itm-5").click()
The correct XPath locator would be as simple as:
//span[text()='DPD Trend']
If you will still not be able to locate the element:
double check that it is not in an iframe, if it is - you will need to switch to it prior to finding
double check that it is not hidden in the shadow DOM, if it is - you will need to locate ShadowRoot via execute_script function and cast the result to a WebElement
a good practice is using WebDriverWait for locating the elements as it might be the case they're not immediately available in DOM, check out How to use Selenium to test web applications using AJAX technology article for more details.
I am trying to scrape content of a page.
Let's say this is the page:
http://finance.yahoo.com/quote/AAPL/key-statistics?p=AAPL
I know I need to use Selenium to get the data I want.
I found this example from Stackoverflow that shows how to do it:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("http://finance.yahoo.com/quote/AAPL/profile?p=AAPL")
# wait for the Full Time Employees to be visible
wait = WebDriverWait(driver, 10)
employees = wait.until(EC.visibility_of_element_located((By.XPATH, "//span[. = 'Full Time Employees']/following-sibling::strong")))
print(employees.text)
driver.close()
My question is this:
In the above example to find Full Time Employees the code that has been used is:
employees = wait.until(EC.visibility_of_element_located((By.XPATH, "//span[. = 'Full Time Employees']/following-sibling::strong")))
How the author has found that s/he needs to use:
"//span[. = 'Full Time Employees']/following-sibling::strong"
To find the number of employees.
For my example page: http://finance.yahoo.com/quote/AAPL/key-statistics?p=AAPL how can I find for example Trailing P/E?
Can you please tell me the steps you took to find this? I do right click and choose Inspect, but then what shall I do?
A picture is worth of thousand words.
In web dev. tools (F12) you do the following steps:
Choose Elements tab
Press Element Selector button
With that button pressed you click an element in the main browser window.
In the DOM-elements window you right-click that highlighted element.
The context menu gets transpired and you choose Copy.
Choose Copy XPath in a sub menu. Now you have that element xpath in a console buffer.
NOTE!
The browser makes/composes an element xpath based on its own algorithm. It might not be the way you think or the way that fits to your code. So, you have to understand xpath in nature, be acquainted with it.
See what xpath the Chrome browser has issued for Trailing P/E:
//*[#id="main-0-Quote-Proxy"]/section/div[2]/section/div/section/div[2]/div[1]/div[1]/div/table/tbody/tr[3]/td[1]/span
'//h3[contains(., "Valuation Measures")]/following-sibling::div[1]//tr[3]'
Here I have the answer for all your confusions.
It will be better to look on some xpath tutorials and do practice from yourself, then you will be able to decide what you have to use .
There are so many site. You can start Here or Here
Now come to your Query -
Suppose I am using following xpath to locate the element
//h3/span[text()='Financial Highlights']/../preceding-sibling::div//tr[3]/td/span
Your requirement to find Trailing P/E in your page, definatly you will look unique xpath which won't change. If you try to find this using firepath it shows some lengthy xpath
Now you will check alternative and find another element (may be sibling, child or ancestor of your element) based on that you can to locate your element
in My case, first will find the Financial Highlights text which I will be able to find using //h3/span[text()='Financial Highlights']
Now I move its parent tag which is h3 and I will do this using /..
I have Trailing P/E element in just above the current node so move on just above node using /preceding-sibling::div
And finally find your element in that <div> like -//tr[3]/td/span
See the screens as well -
Step 1 :
Step 2 :
Step 3 :
Step 4 :