Selenium can not click on one element - python-3.x

I am using Selenium Chromedriver and my problem looks like this:
My program is clicking element on website and a list of elements appear.
Than it is choosing one of element from list using checkbox.
Nextly it is trying to click other element but the program crashes.
When the mentioned list appear if I am clicking anywhere on website using inspect option the code of inspected element looking like this:
<div id="__selectOverlay" style="display: block;"></div>
how can I click any element from website after the list appear?

looking at the html snippet shared, "__selectOverlay" might be the overlay which comes on top of the web page(like a loading spinwheel/panel) .
To get rid of the error, you can try the following
* Wait for panel/overlay to disappear
* Close the panel , if there is a panel close button.
And then click on any element on the web page

As per the HTML you have shared the WebElement seems to me as a Overlay within a div tag. Moreover this div tag doesn't have a onClick() event associated. So we won't be able to click the WebElement straight away. Possibly you might be trying to invoke click() method on some other element which is having this temporary / permanent Overlay through the div tag.
Solution
Depending on the type of Overlay there are two possible solutions as follows :
Temporary Overlay :
In this case induce ExplicitWait with ExpectedConditions set to invisibilityOfElementLocated for the Overlay to be invisible.
WebDriverWait wait1 = new WebDriverWait(driver, 10);
wait1.until(ExpectedConditions.invisibilityOfElementLocated(By.xpath("ele_to_inv")));
Permanent Overlay :
Use JavascriptExecutor to send the click directly on the element.
WebElement ele = driver.findElement(By.xpath("element_xpath"));
JavascriptExecutor executor = (JavascriptExecutor)driver;
executor.executeScript("arguments[0].click();", ele);

Related

Is there a way to scroll up a particular div element in a webpage using selenium with PYTHON?

I could scroll down or up a whole web page but I am having trouble in scrolling a particular div element within the webpage
The thing is when I open a webpage like whatsapp web and within a particular chat containing the messages(which is particular div element),I want to scrape all the messages from the beginning of the chat ,but I could only scrape the messages which is in the view (the last few messages),So I want to scroll that particular div element to the top of the chat to scrape all the messages.
Can someone help me with this thing in PYTHON.
Thank you.
Yes, there are multiple ways using which you could scroll to particular element
By using the move_to_element()
element = driver.find_element_by_xpath("_element_")
action = ActionChains(driver)
action.move_to_element(element).perform()
By scrollIntoView()
element = driver.find_element_by_xpath("_element_")
driver.execute_script("arguments[0].scrollIntoView(true);", element)
For reference check here
It is possible.
Let's say the element you need to scroll inside is div_el, it's locator is xpath_locator the code will be like this:
div_locator = "xpath_locator"
div_el = driver.find_element_by_xpath(div_locator)
driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", div_el)
See more here

How to find elements in containers that open when buttons are pressed

I am using headless Firefox on Selenium and XPath Helper to identify insanely long paths to elements.
When the page initially loads, I can use XPath Helper to find the xpath of any element of interest, and selenium can find the element when given the xpath.
However, several buttons that I need to interact with on the page open menus when pressed that are either small or take up the whole "screen". No matter their size, these containers are overlaid on the original page, and although I can find their xpaths using XPath Helper, when I try to use those xpaths to find the elements using selenium, they can't be found.
I've checked, and there's no iframe funny business happening. I'm a bit stumped as to what could be happening. My guess is that the page's source code is being dynamically changed after I press the buttons that open the menu containers and when I call find_element_by_xpath on new elements in the containers, the original source is being searched, instead of the new source. Could that be it?
Any other ideas?
As a workaround, I can get around this issue by sending keystrokes to the body of the page, but I feel this solution is rather brittle and likely to fail. Would be a much more robust solution to actually specify all elements.
EDIT:
With selenium I can find the export button, but not the menu it opens.
Here is the code for the export button itself:
The element of interest for me is "Customize Export" which I have not been able to find using selenium. Here is the code for this element:
Notice the very top line of this last image (cdk-overlay-container)
Now, when I refresh the page and do NOT click the export button, the cdk-overlay-container section of the code is empty:
This suggests my that my hypothesis is correct -- that when the page loads initially, the "Customize Export" button is nowhere in the source code, but appears only after "Export" is clicked, and that selenium is using the original source code only --not the dynamically generated code that appears after clicking "Export" -- to find elements
Selenium could find the dynamic content after doing
driver.execute_script("return document.body.innerHTML")
The WebDriverWait is what you need to use to wait for a certain condition of elements. Here is an example of waiting for the elements to be clickable before the click with a timeout in 5 seconds:
wait = WebDriverWait(driver, 5)
button = wait.until(EC.element_to_be_clickable((By.XPATH, 'button xpath')))
button.click()
wait.until(EC.element_to_be_clickable((By.XPATH, 'menu xpath'))).click()
identify insanely long paths
is an anti pattern. You can try to not use XPath Helper and find xpath or selector yourself.
Update:
wait = WebDriverWait(driver, 10)
export_buttons = wait.until(EC. presence_of_all_elements_located((By.XPATH, '//button[contains(#class, "mat-menu-trigger") and contains(.="Export")]')))
print("Export button count: ", len(export_buttons))
export_button = wait.until(EC.element_to_be_clickable((By.XPATH, '//button[contains(#class, "mat-menu-trigger") and contains(.="Export")]')))
export_button.click()
cus_export_buttons = wait.until(EC. presence_of_all_elements_located((By.XPATH, '//button[contains(#class, "mat-menu-item") and contains(.="Customize Export")]')))
print("Customize Export button count: ", len(cus_export_buttons))

How to scroll a webpage using selenium webdriver in Python without using javascript method execute_script()

I am trying to scroll a web page by using mouse and scroll bar. I am exploring any other option than
"driver.execute_script("window.scrollBy(0, 5000'))"
I did try options like chrome actions, however nothing seems to be working.
Would need some guidance if anyone has any idea how to solve this.
If your usecase is to scroll() the window containing the DOM document, there is no better way other then using the either of the following Window Methods:
Window.scrollBy()
Window.scrollTo()
If your usecase is to scroll() an Element there is no better way other then using the Element Method:
Element.scrollIntoView()
You can find a detailed discussion in What is the difference between the different scroll options?
However, if you want to avoid the execute_script() to interact with a WebElement you have two (2) other options available as follows:
Using move_to_element() from selenium.webdriver.common.action_chains. This method will automatically scroll the element within the Viewport.
Example code:
menu = driver.find_element_by_css_selector(".nav")
hidden_submenu = driver.find_element_by_css_selector(".nav #submenu1")
ActionChains(driver).move_to_element(menu).click(hidden_submenu).perform()
Using element_to_be_clickable() from selenium.webdriver.support.expected_conditions. This expected_conditions when used in conjunction with selenium.webdriver.support.wait will automatically scroll the element within the Viewport.
Example code:
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "Cart"))).click()

How to access a button inside a <div> container using selenium in python

I want to access a button inside a 'div' tag but the problem is, there are two 'div' tags with same class name and one of them has the button. So how to identify which one has the button and access it. So far I tried to solve but I always get 'Unable to locate element' for button.
One is:
div class="weEq5" style="will-change; width;"
Another one is:
div class="weEq5"
button class="_35EW6"
To find div contain button, first you need to select the button then go to parent element which is div
driver.find_elements_by_xpath('//button/parent::div[#class="weEq5"]')
# or
driver.find_elements_by_xpath('//button[#class="_35EW6"]/parent::div[#class="weEq5"]')
To access the button inside the <div> you can use either of the Locator Strategies:
Using CSS_SELECTOR:
myElement = driver.findElement(By.cssSelector("div[class]:not(style)>button"))
Using XPATH:
myElement = driver.find_element_by_xpath("//div[#class and not (#style)]/button")

Unable to find element in Selenium WebDriver By Name and XPath

I am working with Selenium WebDriver and wrote a small code to find and element (i.e a button) and click on it.
Here is the HTML code for the button:
<input type="submit" name="j_id0:j_id2:j_id3:j_id4:j_id7" value="New Master Health Program" onclick="AddLink()" class="btn">
Here is the C# code for the Test Case:
IWebElement newMasterHealthProgramsLink = driver.FindElement(By.Name("j_id0:j_id2:j_id3:j_id4:j_id7"));
newMasterHealthProgramsLink.Click();
I tried using XPath as well:
IWebElement newMasterHealthProgramsLink = driver.FindElement(By.XPath("//input[#id='j_id0:j_id2:j_id3:j_id4:j_id5']"));
newMasterHealthProgramsLink.Click();
I found a solution saying that you must not have implemented Wait for this. Page does not wait to load completely and tries to find the element. So I added wait command but nothing useful happened. Still getting the same error:
TestAutomation.Driver.Login:
OpenQA.Selenium.NoSuchElementException : The element could not be found
Since your element is in an IFrame, you'll need to 'switch' to that IFrameusing the WebDriver API.
By default, Selenium will use the 'top' frame, and therefore any 'find element' queries will be directed to the most top frame, and ignore any child IFrames.
To solve this, 'switching' to the current IFrame directs Selenium to shove any requests to that frame instead.
driver.SwitchTo().Frame()
Note that you'll need a way of accessing the frame, either by it's ID, index (the top frame is 0, next frame down is 1, etc...) or name.
Another note is that any further requests will be directed to that IFrame, ignoring any others, which includes the top one...so if you need to access the top frame, you'll need to switch back:
driver.Switch().DefaultContent().

Resources