Scrape data from show more button on same page - python-3.x

Question

From website need to scrape odds. At the end of the row have a button on which need to click and from there to scrape odds.
Here is an example of a row for one match
<div class="event-row" event="[object Object]" number-of-displayed-markets="3">
<div class="event-container">
<div class="event-wrap">
<div class="event-wrap-time timelist"> 04:00 </div>
<div class="event-wrap-category has-tooltip" data-original-title="null">
<div class="flag-wrap">
<div class="flags-0-5625">
<div class="flag flag-mex"></div>
</div>
</div>
<div class="tournament-abbreviation"> MEX </div>
</div>
<div class="event-wrap-name pointer"><span>Puebla - Guadalajara</span></div>
<!----><!---->
</div>
<div class="event-outcomes-wrap">
<div class="event-outcomes-row">
<div class="event-outcomes-container">
<!---->
<div class="event-outcomes-odd pointer"> 2.50 </div>
<div class="event-outcomes-odd pointer"> 2.90 </div>
<div class="event-outcomes-odd pointer"> 3.10 </div>
</div>
<!---->
</div>
<div class="event-outcomes-row">
<div class="event-outcomes-container">
<!---->
<div class="event-outcomes-odd pointer"> 1.40 </div>
<div class="event-outcomes-odd pointer"> 1.60 </div>
<div class="event-outcomes-odd pointer"> 2.85 </div>
<div class="event-outcomes-odd pointer"> 5.85 </div>
</div>
<!---->
</div>
<div class="event-outcomes-row">
<div class="event-outcomes-container">
<!---->
<div class="event-outcomes-odd pointer"> 2.25 </div>
<div class="event-outcomes-odd pointer"> 6.30 </div>
<div class="event-outcomes-odd pointer"> 3.65 </div>
<div class="event-outcomes-odd pointer"> 6.25 </div>
</div>
<!---->
</div>
</div>
<div class="event-markets-count pointer">
<div class="event-markets-count-value">+56</div>
</div>
</div>
</div>
After clicking on the button <div class="event-markets-count-value">+56</div> got something like a popup window on the same page and from there I need to scrape <div class="eventview-market-name"> D C </div> odds 1X X2 and 12. I need to do same for every match on page.
<div class="eventview-container noselect">
<div class="eventview-content">
<div class="eventview-header" event="[object Object]">
<div class="eventview-header-content">
<div class="eventview-header-event-name">
<div class="home-team-name">Puebla</div>
<div class="team-names-separator"> - </div>
<div class="away-team-name">Guadalajara</div>
</div>
<div class="eventview-header-details">
<!----><span> Sun 21.11. 04:00 - </span><!----><span>Mexico 1</span>
</div>
<!---->
</div>
<i class="icon icon-close"></i>
</div>
<div class="eventview-bet-groups">
<div class="eventview-bet-groups-button active"><span> All </span></div>
<div class="eventview-bet-groups-button"><span> Basics </span></div>
<div class="eventview-bet-groups-button"><span> Goals </span></div>
<div class="eventview-bet-groups-button"><span> Both teams to score </span></div>
<div class="eventview-bet-groups-button"><span> Combo </span></div>
<div class="eventview-bet-groups-button dropdown">
<span> More </span><i class="icon dropdown-icon icon-arrow-down-a"></i><!---->
</div>
</div>
<div class="eventview-body-wrap">
<!---->
<div class="eventview-market-wrap">
<div class="eventview-market-wrap">
<!---->
<div class="event-markets-wrap">
<div class="eventview-market">
<div market="[object Object]" class="eventview-market-header pointer">
<div class="eventview-market-name"> Finish </div>
<div class="eventview-icons-group">
<div class="market-description-tooltip has-tooltip" data-original-title="null"><i class="icon icon-info-secondary"></i></div>
<div class="favourite-market"><i class="icon icon-star"></i></div>
</div>
</div>
<div class="eventview-outcome-wrapper eventview-market-outcomes expanded">
<div class="eventview-outcome" outcome="[object Object]" marketoutcomeslength="3" market="[object Object]" selectoutcome="function () { [native code] }">
<div class="eventview-outcome-name">1</div>
<div class="eventview-outcome-odd"> 2.50 </div>
</div>
<div class="eventview-outcome" outcome="[object Object]" marketoutcomeslength="3" market="[object Object]" selectoutcome="function () { [native code] }">
<div class="eventview-outcome-name">X</div>
<div class="eventview-outcome-odd"> 2.90 </div>
</div>
<div class="eventview-outcome" outcome="[object Object]" marketoutcomeslength="3" market="[object Object]" selectoutcome="function () { [native code] }">
<div class="eventview-outcome-name">2</div>
<div class="eventview-outcome-odd"> 3.10 </div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="eventview-market-wrap">
<div class="eventview-market-wrap">
<!---->
<div class="event-markets-wrap">
<div class="eventview-market">
<div market="[object Object]" class="eventview-market-header pointer">
<div class="eventview-market-name"> D C </div>
<div class="eventview-icons-group">
<div class="market-description-tooltip has-tooltip" data-original-title="null"><i class="icon icon-info-secondary"></i></div>
<div class="favourite-market"><i class="icon icon-star"></i></div>
</div>
</div>
<div class="eventview-outcome-wrapper eventview-market-outcomes expanded">
<div class="eventview-outcome" outcome="[object Object]" marketoutcomeslength="3" market="[object Object]" selectoutcome="function () { [native code] }">
<div class="eventview-outcome-name">1X</div>
<div class="eventview-outcome-odd"> 1.34 </div>
</div>
<div class="eventview-outcome" outcome="[object Object]" marketoutcomeslength="3" market="[object Object]" selectoutcome="function () { [native code] }">
<div class="eventview-outcome-name">12</div>
<div class="eventview-outcome-odd"> 1.40 </div>
</div>
<div class="eventview-outcome" outcome="[object Object]" marketoutcomeslength="3" market="[object Object]" selectoutcome="function () { [native code] }">
<div class="eventview-outcome-name">X2</div>
<div class="eventview-outcome-odd"> 1.50 </div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="event-markets-wrap">
<div class="eventview-market">
<div market="[object Object]" class="eventview-market-header pointer">
<div class="eventview-icons-group">
<div class="market-description-tooltip has-tooltip" data-original-title="null"><i class="icon icon-info-secondary"></i></div>
<div class="favourite-market"><i class="icon icon-star"></i></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<!---->
</div>
My not working code:
box = driver.find_element(By.XPATH, '//*[#id="timelist-events-wrap"]')
single_row_events = box.find_elements(By.CLASS_NAME, 'event-row')
for match in single_row_events:
more_odds = match.find_element(By.CLASS_NAME, 'event-markets-count').click()
time.sleep(1)
goals = match.find_element(By.XPATH, '//*[#id="app"]/div/div/div[1]/div[2]/div[2]/div[2]/div/div/div[2]/div[4]/span').click()
odds_event = match.find_element(By.XPATH, '//*[#id="app"]/div/div/div[1]/div[2]/div[2]/div[2]/div/div/div[3]/div/div[1]/div/div[1]/div[3]')
odds_events.append(odds_event)
close = match.find_element(By.XPATH, '//*[#id="app"]/div/div/div[1]/div[2]/div[2]/div[2]/div/div/div[1]/i').click()
for odds_event in teams:
for odds_event in odds_events:
for n, box in enumerate(odds_event):
rows = box.find_elements(By.XPATH, './.')
if n == 0:
dc.append(rows[0].text)
This clicking on more_odds but can't scrape and store in an array.
Show me error Traceback (most recent call last): File "C:\Users\Pete\OneDrive\Documents\Scrape\bets\bbet.py", line 57, in <module> more_odds = match.find_element(By.CLASS_NAME, 'event-markets-count').click()

Answer 1

Xpath that you've used isn't relative, they are absolute. Making them brittle in nature. I have corrected a few locators, and I have also induce WebDriverWait (ExplicitWait)
Code:
wait = WebDriverWait(box, 30)
odds_events = []
single_row_events = box.find_elements(By.XPATH, "//div[#class='event-row']")
print("How many rows ?", len(single_row_events))
for match in single_row_events:
try:
time.sleep(.5)
match.find_element(By.XPATH, ".//div[#class='event-markets-count pointer']").click()
print('Row should have clicked')
odds_event = wait.until(EC.visibility_of_element_located((By.XPATH, "//div[#class='away-team-name']"))).text
odds_events.append(odds_event)
except:
pass