I'm trying to click on a link inside a dropdown menu, but I keep getting a TimeoutException, using XPATH.
test = WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div/div[1]/div[2]/div/ul/li[4]/ul/li[7]/a")))
driver.execute_script("arguments[0].click();", test)
But if I try to click on the Log Out option the script sometimes works.
logOut = WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div/div[1]/div[2]/div/ul/li[7]/ul/li/a")))
driver.execute_script("arguments[0].click();", logOut)
There are 3 dropdown menus. The test is in a menu with some 12 options and the Log Out is in another by itself. I want to understand what I'm doing wrong.
Here's the code for the menus.
<body>
<div class="container-fluid">
<div class="row headerLogo">
<!--<header class="navbar-default navbar-static-top headerLogo">-->
<div class="col-md-2">
<div class="vericaltext">WIISCPRD23V</div>
<img src="/Content/images/Logo_menu2.png" class="logoSea" alt='SEA' />
</div>
<div class="col-md-10">
<div class="row flat-nav">
<li class="color20 effect3">
<a><i class="fa fa-comments-o fa-2x"></i><span>Peq</span></a>
<ul class="column-based">
<li class="color20" style="font-weight:bold">Question</li>
<li>Quest</li>
<li>Retr</li>
<li class="color20" style="font-weight:bold">Vitss</li>
<li>Vit</li>
<li class="color20" style="font-weight:bold">BC</li>
<li>Trat</li>
<li>BC</li>
<li class="color20" style="font-weight:bold">CAD</li>
<li>Ant</li>
<li class="color20" style="font-weight:bold">Fer</li>
<li>Add</li>
<li>Emp</li>
<li>Est</li>
<li>Seg</li>
<li>Rec</li>
<li>Cal</li>
</ul>
</li>
<li class="color49 effect3">
<a><i class="fa fa-envelope-o fa-2x"></i><span>E-mails</span></a>
<li class="color5 effect3 divLogout">
<a>
<div style="width:100%;padding-top:15px">
<div class="divAlinhadaEsquerda"><i class="fa fa-user-circle fa-3x"></i></div>
<div class="divAlinhadaEsquerda">
<div class="fontNomUsr">xxx</div>
<div class="fontUser">xxx
</div>
<div class="fontUlti">xxx</div>
</div>
</div>
</a>
<ul class="column-based">
<li><i class="fa fa-power-off"></i><span>Sair</span></li>
</ul>
</li>
</ul>
Your xpath is incorrect, please try below solution :
Solution 1:
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
PesquisasElement=WebDriverWait(browser, 10).until(
EC.visibility_of_element_located((By.XPATH, "//*[contains(text(), 'Pesquisas')]")))
TratamentoElement=WebDriverWait(browser, 10).until(
EC.element_to_be_clickable((By.XPATH, "//a[#href='/ManterBC']")))
#Create the object for Action Chains
actions = ActionChains(driver)
actions.move_to_element(PesquisasElement).move_to_element(TratamentoElement).click()
actions.perform()
Related
I'm using selenium to scrape a web page to get product model numbers. The page has two sections of a grid of products with a card between the two sections. I can grab the model numbers from the first section from "browse-search-pods-1" but I can't access the elements on the bottom half of the page from the second section after "browse-search-pods-2". It ignores the second section. There are 24 products but it only grabs the first 12 from the first section. How can I access both sections?
Here's the website:
https://www.homedepot.com/b/Building-Materials-Drywall/N-5yc1vZar3d?catStyle=ShowProducts
Here's a sample of the html for one product:
<div class="grid">
<section id="browse-search-pods-1" class="grid">
<div class="browse-search__pod col__true-12 col__6-12--xs col__4-12--sm col__3-12--md col__3-12--lg" data-lg-name="Product Pod: 0">
<div class="desktop product-pod" data-automation-id="podnode" data-type="product">
<div class="product-pod--padding">
More Options
<div class="product-pod__title product-pod__title__product">
<a href="/p/USG-Sheetrock-Brand-1-2-in-x-4-ft-x-8-ft-UltraLight-Drywall-14113411708/202530243" class="header product-pod--ie-fix">
<div class="product-pod--ie-fix product-pod__title-control">
<h2 class="product-pod__title product-pod__title__product"><span class="product-pod__title__brand--bold">USG Sheetrock Brand</span><span class="product-pod__title__product">1/2 in. x 4 ft. x 8 ft. UltraLight Drywall</span></h2>
</div>
</a>
</div>
<div class="ratings-and-model-number-container">
<div class="product-pod-list__identifiers">
<div class="product-identifier product-identifier__model">Model# 14113411708</div>
</div>
<a href="/p/USG-Sheetrock-Brand-1-2-in-x-4-ft-x-8-ft-UltraLight-Drywall-14113411708/202530243#ratings-and-reviews" data-testid="product-pod__ratings-link">
<div class="ratings--6r7g3">
<div class="reviews--c43xm reviews--no-margin--c43xm" title=""><span class="stars--c43xm" style="width:89.80600000000001%"></span></div>
<span class="ratings__count--6r7g3">
(<!-- -->3753<!-- -->)
</span>
</div>
</a>
</div>
</div>
</div>
</div>
</section>
<section id="browse-search-pods-2" class="grid">
<div class="category-cards col__12-12" data-lg-name="Product Pod: 0">
<div class="category-cards__zone-wrapper category-cards__zone-card">
<section class="zone-card__zone1">
<div class="zone-card__header-wrapper">
<h2 class="zone-card__header u__bold">Project Guide</h2>
<p class="zone-card__header-text">Installing Drywall Project Guide</p>
</div>
<div class="zone-card-details">
<div class="zone-card-details__image"><img src="https://www.homedepot.com/hdus/en_US/DTCCOMNEW/fetch/FetchRules/FetchPN/how-to-install-drywall-professional-steps-HT-PG-BM.jpg" alt="" class="stretchy" height="1" width="1" loading="lazy"></div>
<div class="zone-card-details__description">
<div class="zone-card-details__text category-cards-details__text--truncate">Hanging drywall is not difficult if you have patience, the right tools and a friend to help. Follow our instructions to learn more</div>
<div class="zone-card-details__actions"><a class="bttn-outline bttn-outline--primary bttn--inline zone-card-details__btn" href="//www.homedepot.com/c/how_to_install_drywall_professional_steps_HT_PG_BM"><span class="bttn__content">Read Our Guide</span></a></div>
</div>
</div>
</section>
<section class="zone-card__zone2">
<div class="zone-card__header-wrapper">
<h2 class="u__truncate zone-card__header u__bold">Buying Guide</h2>
<p class="zone-card__header-text">Types of Drywall</p>
</div>
<div class="zone-card__video-wrapper">
<a class="zone-card__vidcap-link" href="//www.homedepot.com/c/ab/types-of-drywall/9ba683603be9fa5395fab90c24feaae">
<div class="zone-card-details__image zone-card-details__image--vidcap" style="background-image: url("https://i3.ytimg.com/vi/4hF9_z3IqaA/mqdefault.jpg");"></div>
</a>
</div>
<a class="zone-card__video-link" href="//www.homedepot.com/c/ab/types-of-drywall/9ba683603be9fa5395fab90c24feaae">See Our Tips</a>
</section>
</div>
</div>
<div class="browse-search__pod col__true-12 col__6-12--xs col__4-12--sm col__3-12--md col__3-12--lg">
<div class="desktop product-pod" data-automation-id="podnode" data-type="product">
<div class="product-pod--padding">
More Options
<div class="product-pod__title product-pod__title__product">
<a href="/p/Westpac-Materials-18-lb-Fast-Set-20-Lite-Setting-Type-Joint-Compound-22165H/100320411" class="header product-pod--ie-fix">
<div class="product-pod--ie-fix product-pod__title-control">
<h2 class="product-pod__title product-pod__title__product"><span class="product-pod__title__brand--bold">Westpac Materials</span><span class="product-pod__title__product">18 lb. Fast Set 20 Lite Setting-Type Joint Compound</span></h2>
</div>
</a>
</div>
<div class="ratings-and-model-number-container">
<div class="product-pod-list__identifiers">
<div class="product-identifier product-identifier__model">Model# 22165H</div>
</div>
<a href="/p/Westpac-Materials-18-lb-Fast-Set-20-Lite-Setting-Type-Joint-Compound-22165H/100320411#ratings-and-reviews" data-testid="product-pod__ratings-link">
<div class="ratings--6r7g3">
<div class="reviews--c43xm reviews--no-margin--c43xm" title=""><span class="stars--c43xm" style="width: 94.16%;"></span></div>
<span class="ratings__count--6r7g3">(226)</span>
</div>
</a>
</div>
</div>
</div>
</div>
</section>
</div>
Here's the code I've tried to access the second section but I get the model numbers from the first:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
options = Options()
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
driver.get('https://www.homedepot.com/b/Building-Materials-Drywall/N-5yc1vZar3d?catStyle=ShowProducts')
section_two = driver.find_element(By.XPATH, "//section[contains(#id, 'browse-search-pods-2')]")
product_model = section_two.find_elements(By.XPATH, "//div[contains(#class, 'product-identifier product-identifier__model')]")
for model in product_model:
print(model.text)
Try scrolling to the element browse-search-pods-2 and then do
section_two = driver.find_element(By.XPATH, "//section[contains(#id, 'browse-search-pods-2')]")
For scrolling you can try:
org.openqa.selenium.interactions.Actions are reflected in ActionChains class:
from selenium.webdriver.common.action_chains import ActionChains
element = driver.find_element(By.XPATH, "//section[contains(#id, 'browse-search-pods-2')]")
actions = ActionChains(driver)
actions.move_to_element(element).perform()
Or, you can also "scroll into view" via scrollIntoView():
driver.execute_script("arguments[0].scrollIntoView();", element)
I have the following source code:
<form method="POST" data-component="compareForm" action="#">
<div class="row tsp" data-component="list-page-product">
<article id="123">
<div id='product'>
</div>
<div class="stock">
<ul class="simple" data-product="availability">
<li class="available">
<i class="icon-tick"></i>
<span>Delivery available</span></li>
</ul>
</div>
<div data-component="CT">
<button class="TT" type="button">Visit product</button>
</div>
</article>
<article id="1234">
<div id='product'>
</div>
<div class="stock">
<ul class="simple" data-product="availability">
<li class="available">
<i class="icon-tick"></i>
<span>Delivery available</span></li>
</ul>
</div>
<div data-component="CT">
<button class="TT" type="button">Visit product</button>
</div>
</article>
</div>
</form>
I would like to press the "Visit product" button if I found a class name of "available". In this example only article id="123" should be a match.
My code is:
if self.driver.find_elements_by_xpath("//li[#class='available']"):
self.driver.find_element_by_xpath('//*[#class="TT"]').click()
The first error is that it cannot locate an element using XPath. I don't know what to do next. Any input is much appreciated. Thank you!
If I were you I would search for articles then iterate and if available class is found click.
from selenium import webdriver
d = webdriver.Chrome()
d.get('URL')
articles = d.find_elements_by_xpath('//article')
for article in articles:
try:
available = article.find_element_by_class_name(
'//li[#class="available"]')
article.find_element_by_xpath('//button[#class="TT"]').click()
except:
pass
If button class is not only 'TT' or li class is not only 'available' this will not work. In that case you could use find_element_by_class_name.
I have a problem with Selenium webdriver.
I want to log in a website and collect some articles from there. I could log in the website with my code, but couldn't find all elements.
This code can find all elements except ninth element of 'major' and 'notice'. (I don't know why but it could find ninth element of 'upload')
from selenium import webdriver
import MyCode
Data = []
....
driver = webdriver.Chrome()
driver.get(MyCode.url[0])
driver.implicitly_wait(10)
major = driver.find_elements_by_class_name('board-lecture-title')
notice = driver.find_elements_by_xpath('//*[#class=\'post-title\']/*')
upload = driver.find_elements_by_class_name('post-date')
....
driver.close()
This is a part of HTML code of the website
<li class="isnotice" style="width:calc(100% - 20px) !important;">
<span class="post-title">
<a href="https://url">
<span class="board-lecture-title">[Course]</span>Title
</a>
</span>
<br>
<span class="post-date">2020년 7월 30일, 목요일, 오전 10:58</span>
<span class="post-viewinfo area-right">
0
<br>
<span>View</span>
</span>
</li>
<li class="isnotice" style="width:calc(100% - 20px) !important;">
<span class="post-title">
<a href="https://url">
<span class="board-lecture-title">[Course]</span>Title
</a>
</span>
<br>
<span class="post-date">2020년 7월 15일, 수요일, 오후 12:20</span>
<span class="post-viewinfo area-right">
0
<br>
<span>View</span>
</span>
</li>
<li class="isnotice" style="width:calc(100% - 20px) !important;">
<span class="post-title">
<a href="https://url">
<span class="board-lecture-title">[Course]</span>Title
</a>
</span>
<br>
<span class="post-date">2020년 6월 29일, 월요일, 오전 11:18</span>
<span class="post-viewinfo area-right">
47
<br>
<span>View</span>
</span>
....
</ul>
I'm sorry for my poor english give you confusion.
wondering how to target the "Switch" text on the below html:
<div class="product_title">
<a href="/game/pc/into-the-breach" class="hover_none">
<h1>Into the Breach</h1>
</a>
<span class="platform">
<a href="/game/pc">
PC
</a>
</span>
</div>
<div class="product_data">
<ul class="summary_details">
<li class="summary_detail publisher" >
<span class="label">Publisher:</span>
<span class="data">
<a href="/company/subset-games" >
Subset Games
</a>
</span>
</li>
<li class="summary_detail release_data">
<span class="label">Release Date:</span>
<span class="data" >Feb 27, 2018</span>
</li>
<li class="summary_detail product_platforms">
<span class="label">Also On:</span>
<span class="data">
Switch </span>
</li>
</ul>
</div>
so far I am capturing the "Also On:" text as well (with a lot of spaces) with this code:
self.playable_on_systems_label.setText(self.html_soup.find("span", class_='platform').text.strip() + ', ' + self.html_soup.find("li", class_='summary_detail product_platforms').text.strip())
how do I capture (in this case) only the "Switch" text?
FYI - for the first half of the statement (capturing the "PC") text works fine just not the "also on" text
Thanks in advance,
Your query is getting the entire span element with class="summary_detail product_platforms", which is going to include all the text starting from "Also On:" until "Switch." Try something like .find('a', href=re.compile("^.+switch.+$")) or alternately (using CSS) .select("a[href*=switch]") (solution from here)
you can use BeautifulSoup select() function to navigate the the "Switch" text, check this code!!!
rom bs4 import BeautifulSoup
html = '''<div class="product_title">
<a class="hover_none" href="/game/pc/into-the-breach">
<h1>Into the Breach</h1>
</a>
<span class="platform">
<a href="/game/pc">
PC
</a>
</span>
</div>
<div class="product_data">
<ul class="summary_details">
<li class="summary_detail publisher">
<span class="label">Publisher:</span>
<span class="data">
<a href="/company/subset-games">
Subset Games
</a>
</span>
</li>
<li class="summary_detail release_data">
<span class="label">Release Date:</span>
<span class="data">Feb 27, 2018</span>
</li>
<li class="summary_detail product_platforms">
<span class="label">Also On:</span>
<span class="data">
<a class="hover_none" href="/game/switch/into-the-breach">Switch</a> </span>
</li>
</ul>
</div>'''
soup = BeautifulSoup(html, 'html.parser')
text = soup.select('.summary_detail.product_platforms .hover_none')[0].text.strip()
print(text)
Output:
Switch
I done all with selenium and webdriver and now not sure how to get text from ALL div class Text3. But also I have problem with div id="TableStart_00023" That changes now and then numbers "TableStart_00023, TableStart_0283 etc.."
Here is HTML PART OF CODE
<div data-reactroot="" id="TableStart_00023">
<ul>
<li class="FirstRow03">
<a class="aClass">
<div class="innerCl">
<div class="Text1"></div>
<div class="Text2"></div>
<div class="Text3">Wanted data</div>
<div class="Text4"></div>
</div>
</a>
</li>
<li class="FirstRow02">
<a class="aClass">
<div class="innerCl">
<div class="Text1"></div>
<div class="Text2"></div>
<div class="Text3">Wanted data 2</div>
<div class="Text4"></div>
</div>
</a>
</li>
</ul>
</div>
Here is Python PART OF CODE what I done
for content in driver.find_elements_by_id('TableStart_00023'):
mytext= content.find_element_by_xpath('.//div[#class="Text3"]').text
print(mytext)
How can I create loop thought all div class Text3 and get text, when ID TableStart changes numbers? What am I doing wrong?
This xpath will return all elements div class Text3 from your table:
//div[starts-with(#id,'TableStart')]//div[#class='Text3']
When you have all this elements (using driver.find_elements_by_xpath) you can get texts from they.