After a successful search, I need to get the hrefs that appear on the page after the search.
enter = present (driver, By.CSS_SELECTOR, "#search-submit")
driver.execute_script ("arguments[0].click()", enter)
elems = driver.find_elements_by_xpath ("//a[#href]")
# this for loop does not work
for elem in elems:
print(elem.get_attribute ("href"))
I am getting
StaleElementReferenceException: Message: The element reference of is stale
driver.refresh() does not fix the problem. driver.back() will revove the search results. What else could be done?
UPDATE
Florent B's. solution is the way to go. Today I have tested extensively,
works like a charm and faster
enter = self.present (driver, By.CSS_SELECTOR, "#search-submit")
driver.execute_script ("arguments[0].click()", enter)
**WebDriverWait (driver, 20).until (EC.staleness_of (enter))**
Please put a sleep for 30 seconds prior to calling:
elems = driver.find_elements_by_xpath ("//a[#href]")
Glad this helped.
Edit:
You can execute this script via execute_script
"return window.jQuery && jQuery.active == 0"
which will return true IF all ajax requests are done if you prefer to avoid the sleep
Related
I am uploading an image, and I used time.sleep(3) to make it finish upload. But sometimes 3 seconds isn't enough. I am not sure how the code should look like to make slenium to check for a specific tag is shown, then continue to do something.
#input image path
try:
element = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "/html/body/table[2]/tbody/tr[2]/td/h5/ul/li[1]/div[2]/ul/li/form[1]/input[5]")))
element.send_keys('C:\\1.png')
print("Add actionImageURL 1: " + str(element.get_attribute('value')))
except Exception:
print("actionImageURL 1 Path Failed")
#click upload button
try:
element = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "/html/body/table[2]/tbody/tr[2]/td/h5/ul/li[1]/div[2]/ul/li/form[1]/input[6]")))
element.click()
print("actionImageURL 1 Uploading...")
#maybe here is where i can do find for the tag and do a loop wait?
images = driver.find_elements_by_class_name('img-thumbnail ng-hide')
previewlink = images.get_attribute('src')
print(previewlink)
if preview == 't.notInPlaceActionImageURL_preview':
print('Link: ' + print(previewlink))
time.sleep(3)
print("actionImageURL 1 Uploaded")
except Exception:
print("actionImageURL 1 Upload Button NOT FOUND")
I do not know how to deal with html tag like this, see before and after when image is done uploading:
Before:
After:
implicit might solve my problem, but i am looking to workaround with the tag, and how do deal with tag like those. Thanks
If you code a method to wait you can use that wait method in your project
private static WebElement waitForElement(By locator, int timeout)
{
WebElement element=new WebDriverWait(driver,timeout).until(ExpectedConditions.presenceOfElementLocated(locator));
return element;
}
The above code is wait method
If you want to use this method
waitForElement(By.xpath("//button[#type='submit']"),50);
This for example ,you can use your web element here
I created a loop (while True) to automate a task on the site with python. This code clicks on two fields until an element appears on the page
(browser.find_element_by_id ('formComp: buttonBack').
When this element appears, I want the loop to stop and go to the next block of code.
I tested it that way, but it made a mistake. Python reported that the element "formComp: buttonback" was not found. But that's just it, if not found continue the loop:
while (browser.find_element_by_id('formComp:repeatCompromissoLista:0:tableRealizacao:0:subtableVinculacoes:0:vinculacao_input')):
vinc = wait.until(EC.presence_of_element_located((By.ID, 'formComp:repeatCompromissoLista:0:tableRealizacao:0:subtableVinculacoes:0:vinculacao_input')))
vinc = browser.find_element_by_id('formComp:repeatCompromissoLista:0:tableRealizacao:0:subtableVinculacoes:0:vinculacao_input')
vinc.send_keys('400')
enterElem5 = wait.until(EC.element_to_be_clickable((By.ID, 'formComp:buttonConfirmar')))
enterElem5 = browser.find_element_by_id('formComp:buttonConfirmar')
enterElem5.send_keys(Keys.ENTER)
time.sleep(int(segundosv))
if (browser.find_element_by_id('formComp:buttonRetornar')== True):
break
else:
continue
Try like this hope this helps.Check the length count of the button more than 0.
if (len(browser.find_elements_by_id('formComp:buttonRetornar'))>0):
break
else:
continue
find_element_by_id() does not return False when an element is not found. Instead, it raises selenium.common.exceptions.NoSuchElementException. You can handle the exception to get the flow control you are looking for:
try:
browser.find_element_by_id('formComp:buttonRetornar')
break
except NoSuchElementException:
continue
The developer changed the code to use an onclick() DOM element instead of an url. So Now I need to reload page all the time to prevent it from getting stale. How can I do this with a single find_elements_by_xpath?
I assume it is the document.*.submit() that needs the DOM?
https://localhost:4778/ruleengine/AlarmTest?category=Alert#,
text:(Force),
onclick():document.Forceee0deabfba2341d2a0988779499f5530.submit()
Old code now fails:
driver.get(alarmurl)
elems = driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")
for el in elems:
el.click()
My current workaround is to reload the page after every click, but
we may have 3000 events to remove, making it horrendously slow.
driver.get(alarmurl)
elems = driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")
while len(elems) > 0:
driver.get(alarmurl)
elems = driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")
elems[0].click()
Thanks
I dont think you have to reload the entire page if you encounter StaleElementReferenceException but I may be wrong too. It happens When the element is no longer attached to the DOM, search for the element again to reference the element
The below code may not clear your issue, but should help you start implementing a better solution
driver.get(alarmurl)
elems = driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")
for el in elems:
try:
el.click()
except StaleElementReferenceException:
# find the element again and click
You can fix your code as below:
driver.get(alarmurl)
# get initial events number
elems_count = len(driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]"))
# execute click() for each event
for _ in range(elems_count):
driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")[0].click()
P.S. You also might need to wait until events number decreased by 1 after each click in case events removes too slow... or apply another approach:
driver.get(alarmurl)
# get initial events number
elems_count = len(driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]"))
# execute click() for each event. Starting from the last one
for index in range(elems_count):
driver.find_elements_by_xpath("//a[contains(text(), '(Force)')]")[elems_count - index - 1].click()
I am searching elements in my list(one by one) by inputing into searchbar of a website and get apple products name that appeared in search result and printed. However I am getting following exception
StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
I know its because of changing of element very fast so I need to add wait like
wait(driver, 10).until(EC.visibility_of_element_located((By.ID, "submitbutton")))
or explictly
Q1. But I don't understand where should I add it? Here is my code. Please help!
Q2. I want to go to all the next pages using but that's not working.
driver.find_element_by_xpath( '//div[#class="no-hover"]/a' ).click()
Earlier exception was raised on submitton button and now at if statement.
That's not what implicit wait is for. Since the page change regularly you can't be sure when the current object inside the variable is still valid.
My suggestion is to run the above code in loop using try except. Something like the following:
for element in mylist:
ok = False
while True:
try:
do_something_useful_while_the_page_can_change(element)
except StaleElementReferenceException:
# retry
continue
else:
# go to next element
break
Where:
def do_something_useful_while_the_page_can_change(element):
searchElement = driver.find_element_by_id("searchbar")
searchElement.send_keys(element)
driver.find_element_by_id("searchbutton").click()
items_count = 0
items = driver.find_elements_by_class_name( 'searchresult' )
for i, item in enumerate( items ):
if 'apple' in item.text:
print ('item.text')
items_count += len( items )
I think what you had was doing too much and can be simplified. You basically need to loop through a list of search terms, myList. Inside that loop you send the search term to the searchbox and click search. Still inside that loop you want to grab all the elements off the page that consist of search results, class='search-result-product-url' but also the text of the element contains 'apple'. The XPath locator I provided should do both so that the collection that is returned all are ones you want to print... so print each. End loop... back to next search term.
for element in mylist:
driver.find_element_by_id("search-input").send_keys(element)
driver.find_element_by_id("button-search").click()
# may need a wait here?
for item in driver.find_elements_by_xpath( "//a[#class='search-result-product-url'][contains(., 'apple')]" ):
print item.text
I am trying to pull out the names of all courses offered by Lynda.com together with the subject so that it appears on my list as '2D Drawing -- Project Soane: Recover a Lost Monument with BIM with Paul F. Aubin'. So I am trying to write a script that will go to each subject on http://www.lynda.com/sitemap/categories and pull out the list of courses. I already managed to get Selenium to go from one subject to another and pull the courses. My only problem is that there is a button 'See X more courses' to see the rest of the courses. Sometimes you have to click it couple of times that´s why I used while loop. But selenium doesn´t seem to execute this click. Does anyone know why?
This is my code:
from selenium import webdriver
url = 'http://www.lynda.com/sitemap/categories'
mydriver = webdriver.Chrome()
mydriver.get(url)
course_list = []
for a in [1,2,3]:
for b in range(1,73):
mydriver.find_element_by_xpath('//*[#id="main-content"]/div[2]/div[3]/div[%d]/ul/li[%d]/a' % (a,b)).click()
while True:
#click the button 'See more results' as long as it´s available
try:
mydriver.find_element_by_xpath('//*[#id="main-content"]/div[1]/div[3]/button').click()
except:
break
subject = mydriver.find_element_by_tag_name('h1') # pull out the subject
courses = mydriver.find_elements_by_tag_name('h3') # pull out the courses
for course in courses:
course_list.append(str(subject.text)+" -- " + str(course.text))
# go back to the initial site
mydriver.get(url)
Scroll to element before clicking:
see_more_results = browser.find_element_by_css_selector('button[class*=see-more-results]')
browser.execute_script('return arguments[0].scrollIntoView()', see_more_results)
see_more_results.click()
One solution how to repeat these actions could be:
def get_number_of_courses():
return len(browser.find_elements_by_css_selector('.course-list > li'))
number_of_courses = get_number_of_courses()
while True:
try:
button = browser.find_element_by_css_selector(CSS_SELECTOR)
browser.execute_script('return arguments[0].scrollIntoView()', button)
button.click()
while True:
new_number_of_courses = get_number_of_courses()
if (new_number_of_courses > number_of_courses):
number_of_courses = new_number_of_courses
break
except:
break
Caveat: it's always better to use build-in explicit wait than while True:
http://www.seleniumhq.org/docs/04_webdriver_advanced.jsp#explicit-waits
The problem is that you're calling a method to find element by class name, but you're passing a xpath. if you're sure this is the correct xpath you'll simply need to change to method to 'find_element_by_xpath'.
A recommendation if you allow: Try to stay away from these long xpaths and go through some tutorials on how to write efficient xpath for example.