Beautifulsoup problem of scraping text in array - python-3.x

Question

Data=
<div class="dojoxGridView" id="dojox_grid__View_1" role="presentation" style="width: 1900px; height: 721px; left: 1px; top: 0px;" widgetid="dojox_grid__View_1">
<input class="dojoxGridHiddenFocus" dojoattachpoint="hiddenFocusNode" role="presentation" type="checkbox"/>
<input class="dojoxGridHiddenFocus" role="presentation" type="checkbox"/>
<div class="dojoxGridScrollbox" dojoattachpoint="scrollboxNode" role="presentation" style="height: 721px;">
<div class="dojoxGridContent" dojoattachpoint="contentNode" hidefocus="hidefocus" role="presentation" style="height: 504px; width: 1900px;">
<div role="presentation" style="position: absolute; left: 0px; top: 0px;">
<div aria-selected="false" class="dojoxGridRow" role="row" style="">
<table border="0" cellpadding="0" cellspacing="0" class="dojoxGridRowTable" role="presentation" style="width: 1900px;">
<tbody>
<tr>
<td class="dojoxGridCell" idx="0" role="gridcell" style="display:none;width:100px;" tabindex="-1">
78126
</td>
<td class="dojoxGridCell" idx="1" role="gridcell" style="width:10%;" tabindex="-1">
Approved Plan
</td>
<td class="dojoxGridCell" idx="2" role="gridcell" style="width:10%;" tabindex="-1">
G-10
</td>
<td class="dojoxGridCell" idx="3" role="gridcell" style="width:40%;" tabindex="-1">
ROOF PLAN
</td>
</tr>
</tbody>
</table>
</div>
Input=
source = driver.page_source
soup = BeautifulSoup(source, "lxml")
print(soup. prettify())
for article in soup.find_all('div', class_='dojoxGridContent'):
drawing_no = article.find_all('td', class_='dojoxGridCell', idx='3')
# ->need one more line to extract text
print(""drawing_no")
Output=
<td class="dojoxGridCell" idx="3" role="gridcell" style="width:40%;" tabindex="-1">ROOF PLAN</td> ...
I just want to extract "ROOF PLAN" how should I edit my code?
I tried drawing_no.text and drawing_no.value but it said "no attribute".
Thanks for your help!

Answer 1

try followig code
source="""<div class="dojoxGridView" id="dojox_grid__View_1" role="presentation" style="width: 1900px; height: 721px; left: 1px; top: 0px;" widgetid="dojox_grid__View_1">
<input class="dojoxGridHiddenFocus" dojoattachpoint="hiddenFocusNode" role="presentation" type="checkbox"/>
<input class="dojoxGridHiddenFocus" role="presentation" type="checkbox"/>
<div class="dojoxGridScrollbox" dojoattachpoint="scrollboxNode" role="presentation" style="height: 721px;">
<div class="dojoxGridContent" dojoattachpoint="contentNode" hidefocus="hidefocus" role="presentation" style="height: 504px; width: 1900px;">
<div role="presentation" style="position: absolute; left: 0px; top: 0px;">
<div aria-selected="false" class="dojoxGridRow" role="row" style="">
<table border="0" cellpadding="0" cellspacing="0" class="dojoxGridRowTable" role="presentation" style="width: 1900px;">
<tbody>
<tr>
<td class="dojoxGridCell" idx="0" role="gridcell" style="display:none;width:100px;" tabindex="-1">
78126
</td>
<td class="dojoxGridCell" idx="1" role="gridcell" style="width:10%;" tabindex="-1">
Approved Plan
</td>
<td class="dojoxGridCell" idx="2" role="gridcell" style="width:10%;" tabindex="-1">
G-10
</td>
<td class="dojoxGridCell" idx="3" role="gridcell" style="width:40%;" tabindex="-1">
ROOF PLAN
</td>
</tr>
</tbody>
</table>
</div>"""
from bs4 import BeautifulSoup
soup = BeautifulSoup(source,"html.parser")
for article in soup.find_all('div', class_='dojoxGridContent'):
drawing_no = article.find('td', class_='dojoxGridCell', idx='3')
if drawing_no:
print(drawing_no.get_text())

Answer 2

please try below code. But in general if you pass in idx=3 it will only return one single element. If you want to extract text from multiple element you might want to use a more general identifier.
import lxml
from lxml import html
html_string = """
<div class="dojoxGridView" id="dojox_grid__View_1" role="presentation" style="width: 1900px; height: 721px; left: 1px; top: 0px;" widgetid="dojox_grid__View_1">
<input class="dojoxGridHiddenFocus" dojoattachpoint="hiddenFocusNode" role="presentation" type="checkbox"/>
<input class="dojoxGridHiddenFocus" role="presentation" type="checkbox"/>
<div class="dojoxGridScrollbox" dojoattachpoint="scrollboxNode" role="presentation" style="height: 721px;">
<div class="dojoxGridContent" dojoattachpoint="contentNode" hidefocus="hidefocus" role="presentation" style="height: 504px; width: 1900px;">
<div role="presentation" style="position: absolute; left: 0px; top: 0px;">
<div aria-selected="false" class="dojoxGridRow" role="row" style="">
<table border="0" cellpadding="0" cellspacing="0" class="dojoxGridRowTable" role="presentation" style="width: 1900px;">
<tbody>
<tr>
<td class="dojoxGridCell" idx="0" role="gridcell" style="display:none;width:100px;" tabindex="-1">
78126
</td>
<td class="dojoxGridCell" idx="1" role="gridcell" style="width:10%;" tabindex="-1">
Approved Plan
</td>
<td class="dojoxGridCell" idx="2" role="gridcell" style="width:10%;" tabindex="-1">
G-10
</td>
<td class="dojoxGridCell" idx="3" role="gridcell" style="width:40%;" tabindex="-1">
ROOF PLAN
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
</div>
"""
tree = html.fromstring(html_string)
ROOFPLAN = tree.xpath('//tbody/tr//td[#idx="3"]/text()')
print(''.join(ROOFPLAN).strip())

Answer 3

You can use the idx attribute and select by its value
print(soup.select_one("[idx='3']").text.strip())