selenium python onclick() gives StaleElementReferenceException - python-3.x

The developer changed the code to use an onclick() DOM element instead of an url. So Now I need to reload page all the time to prevent it from getting stale. How can I do this with a single find_elements_by_xpath?
I assume it is the document.*.submit() that needs the DOM?
https://localhost:4778/ruleengine/AlarmTest?category=Alert#,
text:(Force),
onclick():document.Forceee0deabfba2341d2a0988779499f5530.submit()
Old code now fails:
driver.get(alarmurl)
elems = driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")
for el in elems:
el.click()
My current workaround is to reload the page after every click, but
we may have 3000 events to remove, making it horrendously slow.
driver.get(alarmurl)
elems = driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")
while len(elems) > 0:
driver.get(alarmurl)
elems = driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")
elems[0].click()
Thanks

I dont think you have to reload the entire page if you encounter StaleElementReferenceException but I may be wrong too. It happens When the element is no longer attached to the DOM, search for the element again to reference the element
The below code may not clear your issue, but should help you start implementing a better solution
driver.get(alarmurl)
elems = driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")
for el in elems:
try:
el.click()
except StaleElementReferenceException:
# find the element again and click

You can fix your code as below:
driver.get(alarmurl)
# get initial events number
elems_count = len(driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]"))
# execute click() for each event
for _ in range(elems_count):
driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")[0].click()
P.S. You also might need to wait until events number decreased by 1 after each click in case events removes too slow... or apply another approach:
driver.get(alarmurl)
# get initial events number
elems_count = len(driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]"))
# execute click() for each event. Starting from the last one
for index in range(elems_count):
driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")[elems_count - index - 1].click()

Related

Refresh a page if an element is not visible and click, else click python selenium

I'm dealing with a very non responsive scroll bar in a window on a tableau dashboard. I cannot scroll to find the element using any code, python, or java execution, so I need to write a condition inside my for loop that refreshes the page, and clicks a location (the scroll bar) if the element isn't present, then checks again for the element's presence, but I can't seem to get it.
for key, value in series_dict.items():
xpath = driver.find_element_by_xpath('//*[#id="' + value + '"]') #how do i make this a boolean
while True:
if xpath == False:
driver.refresh()
#click the scroll bar here then break and start the whole for loop over
else:
try:
series_click = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="' + value + '"]')))
series_click.click()
time.sleep(5)
download_click = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="download"]/span[3]')))
download_click.click()
time.sleep(1)
except TimeoutError as e:
print('Error: {}'.format(str(e)))
continue
break
I've never nested so many conditionals before and I think I've gotten my continues, breaks, and order mixed up. I also don't know how to make my xpath object a boolean so that I can test it.
Any help with this would be appreciated.
Thank you!

Problem with StaleElementReferenceException error pops up occasionally

Hi I have this code which is generating me:
selenium.common.exceptions.StaleElementReferenceException:
Message: stale element reference: element is not attached to the page document
What can be the root cause for it? This code works ok in 10 cases but in 11 not. Is there any chance to improve it to prevent such errors?
Lines which generates the fails are:
compare_announcement_text(context, option_name, element_offer_type)
And:
if element_offer_type.text == option_name:
def select_announcement(context, option_name):
"""Select announcement from 'Offer Type' drop down
:param context:
:param option_name:
:return: Announcement page
"""
offer_type = context.driver.find_element_by_xpath(Locators.offer_type)
offer_type.click()
offer_type_list = context.driver.find_elements_by_xpath(Locators.all_elements_buttons_offer_type)
offer_type_text = []
for element_offer_type in offer_type_list:
compare_announcement_text(context, option_name, element_offer_type)
offer_type_text.append(element_offer_type.text)
if option_name not in offer_type_text:
time.sleep(0.5) # menu not fully rendered - need to wait
offer_type_list = context.driver.find_elements_by_xpath(Locators.all_elements_buttons_offer_type)
for element_offer_type in offer_type_list:
compare_announcement_text(context, option_name, element_offer_type)
from features.pages.announcement_page import AnnouncementPage
return AnnouncementPage(context)
def compare_announcement_text(context, option_name, element_offer_type):
if element_offer_type.text == option_name:
WebDriverWait(context.driver, 20).until(
expected_conditions.visibility_of_all_elements_located(
(By.XPATH, Locators.all_elements_buttons_offer_type)))
for check in range(4):
try:
ActionChains(context.driver).move_to_element(element_offer_type).click(element_offer_type).perform()
return
except selenium.common.exceptions.ElementClickInterceptedException:
time.sleep(0.5)
else:
print(f"Tried click {option_name} for 2 seconds - making final click")
ActionChains(context.driver).move_to_element(element_offer_type).click(element_offer_type).perform()
else:
print(f'Element selected in offer type is not {option_name} and is {element_offer_type.text}')
It's easily cause this problem if you use find_elements_by_xpath to get a set of elements first.
I suggest you find element again in for loop like this:
offer_type_list = context.driver.find_elements_by_xpath(Locators.all_elements_buttons_offer_type)
offer_type_text = []
for i in range(1, len(offer_type_list)+1):
# xpath example: (//div[#class='abc'])[1]
element_offer_type = context.driver.find_element_by_xpath("("+Locators.all_elements_buttons_offer_type+")["+str(i)+"]")
compare_announcement_text(context, option_name, element_offer_type)
offer_type_text.append(element_offer_type.text)
I faced same exception a time ago. Its caused because the element you are interacting is not loaded in the DOM at that moment. You can solve it with a webDriverWait waiting for the element to be displayed.
#Jonx -> It's not the only reason. Question is also about app, #ranger is using. If it's dynamic (with reloading elements e.g. cause new live data) it's also raises this issue. depending on time when data comes in (e.g. after getting reference to element but before performing action). This can be pain in the ass -,-
#Yun - like this idea, but also (if in testing), using Page Object Pattern, good idea is to use views, and then perform actions within try-except blocks, or actually defining Page elements as property methods, so You'll get clean code, and WebDriver will look for element each time call. That eases up issue.
Yeah, I know that this is not so "well-performance" solution, but - as always - trade offs for stable behaviour with live-dynamic apps.

getting href after a search

After a successful search, I need to get the hrefs that appear on the page after the search.
enter = present (driver, By.CSS_SELECTOR, "#search-submit")
driver.execute_script ("arguments[0].click()", enter)
elems = driver.find_elements_by_xpath ("//a[#href]")
# this for loop does not work
for elem in elems:
print(elem.get_attribute ("href"))
I am getting
StaleElementReferenceException: Message: The element reference of is stale
driver.refresh() does not fix the problem. driver.back() will revove the search results. What else could be done?
UPDATE
Florent B's. solution is the way to go. Today I have tested extensively,
works like a charm and faster
enter = self.present (driver, By.CSS_SELECTOR, "#search-submit")
driver.execute_script ("arguments[0].click()", enter)
**WebDriverWait (driver, 20).until (EC.staleness_of (enter))**
Please put a sleep for 30 seconds prior to calling:
elems = driver.find_elements_by_xpath ("//a[#href]")
Glad this helped.
Edit:
You can execute this script via execute_script
"return window.jQuery && jQuery.active == 0"
which will return true IF all ajax requests are done if you prefer to avoid the sleep

Unable to figure out where to add wait statement in python selenium

I am searching elements in my list(one by one) by inputing into searchbar of a website and get apple products name that appeared in search result and printed. However I am getting following exception
StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
I know its because of changing of element very fast so I need to add wait like
wait(driver, 10).until(EC.visibility_of_element_located((By.ID, "submitbutton")))
or explictly
Q1. But I don't understand where should I add it? Here is my code. Please help!
Q2. I want to go to all the next pages using but that's not working.
driver.find_element_by_xpath( '//div[#class="no-hover"]/a' ).click()
Earlier exception was raised on submitton button and now at if statement.
That's not what implicit wait is for. Since the page change regularly you can't be sure when the current object inside the variable is still valid.
My suggestion is to run the above code in loop using try except. Something like the following:
for element in mylist:
ok = False
while True:
try:
do_something_useful_while_the_page_can_change(element)
except StaleElementReferenceException:
# retry
continue
else:
# go to next element
break
Where:
def do_something_useful_while_the_page_can_change(element):
searchElement = driver.find_element_by_id("searchbar")
searchElement.send_keys(element)
driver.find_element_by_id("searchbutton").click()
items_count = 0
items = driver.find_elements_by_class_name( 'searchresult' )
for i, item in enumerate( items ):
if 'apple' in item.text:
print ('item.text')
items_count += len( items )
I think what you had was doing too much and can be simplified. You basically need to loop through a list of search terms, myList. Inside that loop you send the search term to the searchbox and click search. Still inside that loop you want to grab all the elements off the page that consist of search results, class='search-result-product-url' but also the text of the element contains 'apple'. The XPath locator I provided should do both so that the collection that is returned all are ones you want to print... so print each. End loop... back to next search term.
for element in mylist:
driver.find_element_by_id("search-input").send_keys(element)
driver.find_element_by_id("button-search").click()
# may need a wait here?
for item in driver.find_elements_by_xpath( "//a[#class='search-result-product-url'][contains(., 'apple')]" ):
print item.text

selenium - clicking a button

I am trying to pull out the names of all courses offered by Lynda.com together with the subject so that it appears on my list as '2D Drawing -- Project Soane: Recover a Lost Monument with BIM with Paul F. Aubin'. So I am trying to write a script that will go to each subject on http://www.lynda.com/sitemap/categories and pull out the list of courses. I already managed to get Selenium to go from one subject to another and pull the courses. My only problem is that there is a button 'See X more courses' to see the rest of the courses. Sometimes you have to click it couple of times that´s why I used while loop. But selenium doesn´t seem to execute this click. Does anyone know why?
This is my code:
from selenium import webdriver
url = 'http://www.lynda.com/sitemap/categories'
mydriver = webdriver.Chrome()
mydriver.get(url)
course_list = []
for a in [1,2,3]:
for b in range(1,73):
mydriver.find_element_by_xpath('//*[#id="main-content"]/div[2]/div[3]/div[%d]/ul/li[%d]/a' % (a,b)).click()
while True:
#click the button 'See more results' as long as it´s available
try:
mydriver.find_element_by_xpath('//*[#id="main-content"]/div[1]/div[3]/button').click()
except:
break
subject = mydriver.find_element_by_tag_name('h1') # pull out the subject
courses = mydriver.find_elements_by_tag_name('h3') # pull out the courses
for course in courses:
course_list.append(str(subject.text)+" -- " + str(course.text))
# go back to the initial site
mydriver.get(url)
Scroll to element before clicking:
see_more_results = browser.find_element_by_css_selector('button[class*=see-more-results]')
browser.execute_script('return arguments[0].scrollIntoView()', see_more_results)
see_more_results.click()
One solution how to repeat these actions could be:
def get_number_of_courses():
return len(browser.find_elements_by_css_selector('.course-list > li'))
number_of_courses = get_number_of_courses()
while True:
try:
button = browser.find_element_by_css_selector(CSS_SELECTOR)
browser.execute_script('return arguments[0].scrollIntoView()', button)
button.click()
while True:
new_number_of_courses = get_number_of_courses()
if (new_number_of_courses > number_of_courses):
number_of_courses = new_number_of_courses
break
except:
break
Caveat: it's always better to use build-in explicit wait than while True:
http://www.seleniumhq.org/docs/04_webdriver_advanced.jsp#explicit-waits
The problem is that you're calling a method to find element by class name, but you're passing a xpath. if you're sure this is the correct xpath you'll simply need to change to method to 'find_element_by_xpath'.
A recommendation if you allow: Try to stay away from these long xpaths and go through some tutorials on how to write efficient xpath for example.

Resources