I am trying to test downloading a file from a website using selenium on python.
The website has peculiar design where the file name appears as a text element above the button to download the file. There are no specific names or IDs for these buttons. And they are not known to us. So, I can't specify the ID or element name in the code directly.
Here is the HTML snippet:
<div class="col-xs-12 col-sm-12 col-md-12 col-lg-12 noborderBottom semiBolder-label ">
<span data-bind="text: jurisdictionName, attr: { id: jurisdictionId() + '-guides' }, visible: showInList" id="67-guides">Greece</span>
</div>
<div class="clearfix visible-xs"></div>
<div class="col-xs-6 col-sm-6 col-md-5 col-lg-5 text-center">
<div class="greenPDFIcon cursorPointer align-center" data-bind="event: { click: onHighlightClick.bind($data) }, style: { 'visibility': highLightUrl() ? 'visible' : 'hidden' }" style="visibility: visible;"></div>
<span class="taxGuidesText lg-visible md-visible xs-visible" data-bind="style: { 'visibility': highLightUrl() ? 'visible' : 'hidden' }" style="visibility: visible;">Highlights</span>
<!--<span class="taxGuidesText lg-visible md-visible xs-visible" data-bind="visible:showInList">Highlights</span>-->
</div>
Now, I first need to search for the text "Greece" in the above example.
Get it's location on the webpage:
class="col-xs-12 col-sm-12 col-md-12 col-lg-12 noborderBottom semiBolder-label"
Locate the button right below this text - so in the above example that gives me:
class="col-xs-6 col-sm-6 col-md-5 col-lg-5"
And then click on the button:
class="greenPDFIcon cursorPointer align-center"
The thing is, I do not know this "Greece". That comes through input parameter.
I only know that if the input parameter text is found on the webpage, the button will be right below it. And I have to click it to open the pdf file.
How to do that using selenium on python?
So far I have reached:
s=Service(r"driver_path")
browser = webdriver.Edge(service=s)
browser.get('webpage_url')
country = input('Enter a country name: ')
Also, suggest if I should use anything else rather than selenium to do this, as I understand this is more of web-scrapping than automated testing. I also tried beautifulsoup, but the website is not accessible directly through api. Browser access is required.
To locate the element with the country name e.g. Greece and click on the respective element with text as Highlights you can use the following Locator Strategies:
Using XPATH and "Old Style" String Formatting (% Operator):
browser.get('webpage_url')
country = input('Enter a country name: ')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[contains(#class, 'semiBolder-label')]//span[contains(., '%s')]//following::div[2]//span[contains(., 'Highlights')]" % country))).click()
Using XPATH and "New Style" String Formatting (str.format):
browser.get('webpage_url')
country = input('Enter a country name: ')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[contains(#class, 'semiBolder-label')]//span[contains(., '{}')]//following::div[2]//span[contains(., 'Highlights')]".format(country)))).click()
Using XPATH and String Interpolation / f-Strings (Python 3.6+):
browser.get('webpage_url')
country = input('Enter a country name: ')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, f"//div[contains(#class, 'semiBolder-label')]//span[contains(., '{country}')]//following::div[2]//span[contains(., 'Highlights')]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Related
How to select text content from multiple DIV elements using selenium?
On the website I intend to collect information it contains div and span with the same class.
How can I collect this information separately?
I need the contents inside the panel-body div > span of each block
driver.find_element_by_xpath(".//div[#class='panel-body'][1]/span[1]").text
driver.find_element_by_xpath(".//div[#class='panel-body'][1]/span[2]").text
driver.find_element_by_xpath(".//div[#class='panel-body'][1]/span[3]").text
driver.find_element_by_xpath(".//div[#class='panel-body'][2]/span[1]").text
driver.find_element_by_xpath(".//div[#class='panel-body'][2]/span[2]").text
html
<div class="panel-heading">
<h3 class="panel-title">Identificação</h3>
</div>
<div class="panel-body">
<span class="spanValorVerde">TEXT</span><br>
<span style="font-size:small;color:gray">TEXT</span><br>
<br>
<span class="spanValorVerde">TEXT</span>
</div>
</div>
<div class="panel panel-success">
<div class="panel-heading">
<h3 class="panel-title">Situação Atual</h3>
</div>
<div class="panel-body">
<span class="spanValorVerde">TEXT</span> <br>
<span class="spanValorVerde">TEXT</span>
</div>
</div>
I am expecting that "select text" meant "get text".
first for loop:
count = driver.find_elements_by_xpath(".//div[#class='panel-body'][i]")
second for loop with count iteration:
driver.find_element_by_xpath(".//div[#class='panel-body'][i]/span[j]").text
If you search with findElements with ".//div[#class='panel-body'][i]" will give you the total element present, then add another loop for .//div[#class='panel-body'][i]/span[j] and then get text. Hope it helps!
To extract the texts e.g. TEXT, from each <span> using Selenium and python you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR and get_attribute("innerHTML"):
print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.panel.panel-success div.panel-body span.spanValorVerde")))])
Using XPATH and text attribute:
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[#class='panel panel-success']//div[#class='panel-body']//span[#class='spanValorVerde']")))])
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Outro
Link to useful documentation:
get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium
Cant click on the link due to the fact, all of the value I keep trying to select has the same name as well within the tables.
Tried xpath,by link, by text, by css none of it works. My current code:
driver.find_element_by_xpath('//[#id="homeapp"]/div/div/div[3]/div[1]/table/tbody/tr[3]/td[2]/div[1]/div[2]/span[1]/a').click()
Here is the relevant html code:
<tr class="bui-table__row"><td class="peg-table__cell--no-label bui-table__cell" data-heading="ID" scope="row"><a href="../../extranet_ng/manage/index.html?ses=8594ddf0718dec240a0c9e7991e108a8&name_id=11111" class="bui-link bui-link--secondary" target="_blank" data-track-ga="Groups: Home,Active Properties,Property ID">
1111(number i can click aswell)
</a></td> <td class="peg-table__cell--no-label bui-table__cell" data-heading="Name" scope="row"><div class="bui-avatar-block"><div class="peg-_-avatar-shrink bui-avatar"><img src="https://q-xx.bstatic.com/xdata/images/extra/square60/189090859.jpg?k=cc455d4d570f0e0c86a1b8329b16d53fdba96d78677cc87151d6181e45d38ec5&o=" alt="text i want to click" class="bui-avatar__image"></div> <div class="bui-avatar-block__text"><span class="bui-avatar-block__title"><a href="../../extranet_ng/manage/index.html?ses=8594ddf0718dec240a0c9e7991e108a8&name_id=11111" class="peg-property-link bui-link bui-link--secondary" target="_blank" data-track-ga="Groups: Home,Active Properties,Property Name">
Text i want to click
<!----> <!----></a></span> <span class="bui-avatar-block__subtitle"><!----> </span></div></div> <div class="bui-spacer--smaller"></div> <div class="peg-score-bar__inline bui-score-bar"><div class="bui-score-bar__item"><div class="bui-score-bar__header"><h2 class="bui-score-bar__title"></h2> <span class="bui-score-bar__score">93%</span></div> <div class="bui-score-bar__bar"><span data-value="9.3" class="bui-score-bar__value" style="width: 93%;"></span></div></div></div></td> <td class="peg-table__cell--no-label bui-table__cell" data-heading="Location"><div class="peg-address-wrapper"><span class="peg-flag-wrapper"><div class="peg-flag bui-flag"><img src="https://q.bstatic.com/backend_static/common/flags/16/nl/314ce6500532e846e25d6e3a7c824ef17c968446.png" class="bui-flag__flag" style="height: auto;"> <!----></div></span>
TEXT2
</div></td> <td class="peg-table__cell--no-label bui-table__cell" data-heading="Status"><span>Open/Bookable</span></td> <td class="bui-table__cell bui-table__cell--center" data-heading="Arrivals/departures & tomorrow"><span class="peg-counter--arrivals"><a href="../../extranet_ng/manage/search_reservations.html?ses=8594ddf0718dec240a0c9e7991e108a8&name_id=11111" data-track-ga="Groups: Home,Active Properties,Arrivals" target="_blank" class="peg-counter peg-counter--has"><span aria-label="80 unread" class="bui-bubble">
VALUE1
</span></a></span> <span class="peg-counter--departures"><a href="../../extranet_ng/manage/search_reservations.html?ses=8594ddf0718dec240a0c9e7991e108a8&name_id=11111&type=departure" data-track-ga="Groups: Home,Active Properties,Departures" target="_blank" class="peg-counter peg-counter--has"><span aria-label="80 unread" class="bui-bubble">
VALUE2
</span></a></span></td> <td class="bui-table__cell bui-table__cell--center" data-heading="Guest Messages"><a href="../../extranet_ng/manage/messaging_inbox.html?ses=8594ddf0718dec240a0c9e7991e108a8&name_id=11111" data-track-ga="Groups: Home,Active Properties,Guest Messages" target="_blank" class="peg-counter peg-counter--has"><span aria-label="102 unread" class="bui-bubble">
VALUE3
</span></a></td> <td class="bui-table__cell bui-table__cell--center" data-heading="extranet.com Messages"><a href="../../extranet_ng/manage/inbox.html?ses=8594ddf0718dec240a0c9e7991e108a8&name_id=11111" data-track-ga="Groups: Home,Active Properties,extrabnet.com Messages" target="_blank" class="peg-counter"><span aria-label="0 unread" class="bui-bubble">
0
</span></a></td></tr>
If you want to click the link which contains avatar in its class attribute use the following XPath expression:
//div[contains(#class,'avatar')]/descendant::a[contains(text(), 'text i want to click')]
where:
contains() is XPath Function allowing partial match on the attributes values and/or text
descendant - XPath Axis which matches children of the current node and their respective children
last part limits the search to hyperlinks which contain text you're looking for
The desired element is a ReactJS enabled element, so you have to induce WebDriverWait for the element to be clickable() and you can use either of the following solutions:
Using PARTIAL_LINK_TEXT:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "text i want to click"))).click()
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.peg-property-link.bui-link.bui-link--secondary[href*='/extranet_ng/manage/index'][data-track-ga*='Active Properties']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='peg-property-link bui-link bui-link--secondary' and contains(#href, '/extranet_ng/manage/index')][starts-with(#data-track-ga, 'Groups')]"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I´m trying to find a text box with Python and Selenium.. I Tried by_css_selector, bY XPATH, by ID, by name but the message is always the same:
Unable to locate element: #x-auto-225-input
this is the piece of html. I Want to find the textbox to fill it.
<td class="x-table-layout-cell" role="presentation" style="padding: 2px;">
<div role="presentation" class=" x-form-field-wrap x-component" id="x-auto-225" style="width: 150px;"></div>
<input type="text" class=" x-form-field x-form-text " id="x-auto-225-input" name="PURCHASE_ORDER_CODE_NAME" tabindex="0" style="width: 150px;">
</td>
My last attempt was:
pc = browser.find_element_by_css_selector("#x-auto-225-input").click()
pc.send_keys("7555425-1")
Looking at the html, id mentioned can be dynamic, so you can't put the static id in your identifier.
However, as name attribute is present in the html, you can use that to identify your element, like:
browser.find_element_by_name("PURCHASE_ORDER_CODE_NAME").click()
Updated answer as per discussion with the OP
As an iframe is present on the UI, you need to first switch to the iframe and then click on the element.
To switch to iframe you can use:
browser.switch_to.frame(browser.find_element_by_tag_name('iframe'))
and then use:
pc = browser.find_element_by_name("PURCHASE_ORDER_CODE_NAME")
pc.click()
pc.send_keys("7555425-1")
if you want to switch back to the default content, you can use:
browser.switch_to.default_content()
The desired element is a dynamic element so to invoke click() on the element you have to induce WebDriverWait for the element to be clickable and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.x-form-field.x-form-text[id$='-input'][name='PURCHASE_ORDER_CODE_NAME']"))).click();
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#class=' x-form-field x-form-text ' and contains(#id,'-input')][#name='PURCHASE_ORDER_CODE_NAME']"))).click();
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Maybe you can try another "selector" approach. Ex(Javascript):
selenium.By.xpath('//*[#data-icon="edit"]')
driver.findElement(by).click()
Having trouble figuring out how to click the Next button at the bottom of the table on this page:
https://www.zacks.com/stocks/industry-rank/reit-and-equity-trust-other-266/stocks-in-industry
This is what I've tried:
from bs4 import BeautifulSoup
import requests
import csv, random, time
from pandas.io.html import read_html
from selenium import webdriver
from selenium.webdriver.support.ui import Select
url = 'https://www.zacks.com/stocks/industry-rank/reit-and-equity-trust-other-266/stocks-in-industry'
# Open Chrome
driver = webdriver.Chrome()
# Send Chrome to the URL
page = driver.get(url)
# Wait for page to load a few seconds
timeDelay = random.randrange(4, 8)
time.sleep(timeDelay)
# Try to click the darn button
element = driver.find_element_by_xpath('//*[#id="industry_rank_table_next"]')
driver.execute_script("arguments[0].click();", element)
...and
element = driver.find_element_by_xpath('//*[#id="industry_rank_table_next"]')
element.send_keys("\n")
...found from other answers but not working for me. Simply using .click() does not work. I've also tried selecting the button using css_selector, partial_link_text, and class_name but still no success. I've ran into this on a few sites. Any ideas?
To click() on the element with text as Next you have to induce WebDriverWait for the element to be clickable and you can use either of the following solutions:
Using LINK_TEXT:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "Next"))).click()
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.paginate_button next#industry_rank_table_next"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='paginate_button next' and #id='industry_rank_table_next']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
if the element which you have to click have parent element then you can find parent by findElements method and after than you just need to pass the index of the element like 0 or 1 or so on.. and then you can perform click action on that
Want to click on third li from second ul:
<ul id="select-123" style="width: 1180px; display: none;">
<li class="" style="display:none;">
<li class="">
<li class="">
<li class="">
</ul>
<ul id="select-123" style="width: 1180px; display: none;">
<li class="" style="display:none;">
<li class="">
<li class="">
<li class="">
</ul>
Code I am trying is to select third li from second ul which does not work:
driver.findElements(By.css(ul[id*='select-123'])).then(function(elems) {
elems[2].then(function(lis) {
driver.findElement(By.css("ul[id*='select-123'] li:nth-child(3)")).click();
});
});
Another way if you don't wish to add additional wait(s) to your code and just click the button:
using javascript click:
element = driver.find_element_by_xpath("//a[#class='paginate_button next' and #id='industry_rank_table_next']")
driver.execute_script("arguments[0].click();", element)
UPDATE:
I didn't notice this earlier, but
element = driver.find_element_by_xpath('//*[#id="industry_rank_table_next"]')
driver.execute_script("arguments[0].click();", element)
...works as it clicks the link successfully, but gives me a
selenium.common.exceptions.WebDriverException: Message: unknown error: call function result missing 'value'
...error after it clicks the link. So to get around this, I've just added a try/except to handle the error. ie:
try:
element = driver.find_element_by_xpath('//*[#id="industry_rank_table_next"]')
driver.execute_script("arguments[0].click();", element)
except:
pass
...which seems to work. Seems like such a stupid observation on my part, but thank you to everyone for your help. Hopefully something in here will help someone else in the future.
I want to show more page by click a button to run a JavaScript function as below:
<div class="loading" style="display:none;">
<p class="btn blue"><span>さらに表示</span></p>
</div>
I tried the code, it doesn't work, what can I do?
more_info_button = driver.find_element_by_tag_name('a').get('href=javascript:void(0);')
more_info_button.click()
If you want to click link that contains attribute #href equal to "javascript:void(0);", try
more_info_button = driver.find_element_by_xpath('//a[#href="javascript:void(0);"]')
more_info_button.click()
Same with CSS selector:
more_info_button = driver.find_element_by_css_selector('a[href="javascript:void(0);"]')
To locate link by text in preceding paragraph:
more_info_button = driver.find_element_by_xpath('//a[preceding-sibling::p[.="さらに表示"]]')
Update
Try below code to get extended topics list after clicking the button:
from selenium.webdriver.support.ui import WebDriverWait
topics_number = len(driver.find_elements_by_class_name('topics'))
more_info_button.click()
WebDriverWait(driver, 10).until(lambda driver: len(driver.find_elements_by_class_name('topics')) > topics_number)
extended_list = driver.find_elements_by_class_name('topics')