HTML confusion with Python BeautifulSoup - python-3.x

Question

I'm following thenewboston's tutorials on youtube and after compiling my code I get no errors.
I'm trying to print the "Generic Line List" and all the links following that list;can be found at the bottom of this link
http://playrustwiki.com/wiki/List_of_Items
import requests
from bs4 import BeautifulSoup
def trade_spider(max_pages):
page = 1
while page <= max_pages: #makes our pages change everytime
url = 'http://playrustwiki.com/wiki/List_of_Items' + str(page)
source_code = requests.get(url)
plain_text = source_code.text
soup = BeautifulSoup(plain_text) #find all the links in soup or all the titles
for link in soup.findAll('a', {'class': 'a href'}): #links are a for anchors in HTML
href = link.get('href') # href attribute
print(href)
page += 1
trade_spider(1)
I've tried different HTML attributes but I think thats where my confusion starts. I can't find the correct attribute to call for my scraper or I'm calling the wrong attribute.
Please help~
Thanks :)

Answer 1

The idea here would be to find the element that has Generic line list text. Then, find the next ul sibling via find_next_sibling() and get all links inside via find_all():
h3 = soup.find('h3', text='Generic Line List')
generic_line_list = h3.find_next_sibling('ul')
for link in generic_line_list.find_all('a', href=True):
print(link['href'])
Demo:
>>> import requests
>>> from bs4 import BeautifulSoup
>>>
>>> url = 'http://playrustwiki.com/wiki/List_of_Items'
>>> soup = BeautifulSoup(requests.get(url).content)
>>>
>>> h3 = soup.find('h3', text='Generic Line List')
>>> generic_line_list = h3.find_next_sibling('ul')
>>> for link in generic_line_list.find_all('a', href=True):
... print(link['href'])
...
/wiki/Wood_Barricade
/wiki/Wood_Shelter
...
/wiki/Uber_Hunting_Bow
/wiki/Cooked_Chicken_Breast
/wiki/Anti-Radiation_Pills