Scraping multiple links with bs4 and request - python-3.x

Question

I'm new to this please go easy on me. So i want to scrape name and number of multiple pages but only scraping first url. Also the code is scraping only one item either Name Or Phone Number. Name is in "h2" class "c411ListedName" and number is in 'span' class "c411Phone"
import csv
import requests
from bs4 import BeautifulSoup
urls = ['https://www.canada411.ca/search/si/1/kumar/Canada/?pgLen=100','https://www.canada411.ca/search/si/2/kumar/Canada/?pgLen=100']
for url in urls:
response = requests.get(url)
html = requests.get(url).text
soup = BeautifulSoup(response.content, "html.parser")
products = soup.findAll('span','h2', class_=['c411Phone', 'c411ListedName'])
for div in products:
print(div.text)

Answer 1

What happens
Your requesting both urls, but process only the last one, cause your second loop is will not be executed after your first loop is finished.
Solution
Improve your indentation and put the secaonde loop in the first
import csv
import requests
from bs4 import BeautifulSoup
urls = ['https://www.canada411.ca/search/si/1/kumar/Canada/?pgLen=100','https://www.canada411.ca/search/si/2/kumar/Canada/?pgLen=100']
for url in urls:
response = requests.get(url)
html = requests.get(url).text
soup = BeautifulSoup(response.content, "html.parser")
products = soup.findAll('span','h2', class_=['c411Phone', 'c411ListedName'])
for div in products:
print(url,div.text)
Additional solution
Select and process the enclosing <div>
import csv
import requests
from bs4 import BeautifulSoup
urls = ['https://www.canada411.ca/search/si/1/kumar/Canada/?pgLen=100','https://www.canada411.ca/search/si/2/kumar/Canada/?pgLen=100']
data=[]
for url in urls:
response = requests.get(url)
html = requests.get(url).text
soup = BeautifulSoup(response.content, "html.parser")
for row in soup.select('div.c411Listing.jsResultsList > div.listing__row:nth-of-type(1)'):
data.append({
'name':row.h2.a.text,
'number':row.span.text
})
data

Answer 2

You should do two things:
Put the products out of your for as an array to be able to store all urls' content
Use findall separately for each h2 and span
Here is the code:
import requests
from bs4 import BeautifulSoup
urls = ['https://www.canada411.ca/search/si/1/kumar/Canada/?pgLen=100',
'https://www.canada411.ca/search/si/2/kumar/Canada/?pgLen=100']
products_h = []
products_span = []
for url in urls:
response = requests.get(url)
soup = BeautifulSoup(response.content, "html.parser")
products_h.append(soup.findAll('span', class_=['c411Phone']))
products_span.append(soup.findAll('h2', class_=['c411ListedName']))
for i in range(len(urls)):
for span, h in zip(products_span[i], products_h[i]):
print(f'{span.text},{h.text}')
Here is the result that I got:
B Kumar,[phone]
V Roy Kumar,[phone]
C Kumar,[phone]
Ananth Kumar,[phone]
ASHOK KUMAR,[phone]
R Kumar,[phone]
A Kumar,[phone]
S R Kumar,[phone]
R Kumar,[phone]
S & H Kumar,[phone]
V KUMAR,[phone]
Ashwani Kumar,[phone]
S Kumar,[phone]
V Kumar,[phone]
B C M Kumar,[phone]
Vasant Kumar,[phone]
MICHAEL KUMAR,[phone]
M Kumar,[phone]
G Kumar,[phone]
T Kumar,[phone]
A Kumar,[phone]
U Kumar,[phone]
J Kumar,[phone]
Priya Kumar,[phone]
C Kumar,[phone]
M Kumar,[phone]
R Kumar,[phone]
N Kumar,[phone]
Sanjai Kumar,[phone]
R Kumar,[phone]
S Kumar,[phone]
J Kumar,[phone]
P Kumar,[phone]
R Kumar,[phone]
S Kumar,[phone]
Naresh Kumar,[phone]
K Kumar-Telang,[phone]
Poonam Kumar,[phone]
R Kumar,[phone]
A Kumar,[phone]
A Kumar,[phone]
Subodh Kumar,[phone]
R Kumar,[phone]
S Kumar,[phone]
R Kumar,[phone]
N Kumar,[phone]
R Kumar,[phone]
S Kumar,[phone]
Aneal Kumar,[phone]
Unesh Kumar,[phone]
Virender Kumar,[phone]
R Kumar,[phone]
M Kumar,[phone]
S SAMPATH-kUMAR,[phone]
S Kumar,[phone]
R KUMAR,[phone]
C Kumar,[phone]
A Kumar,[phone]
Ram Kumar,[phone]
D Kumar,[phone]
R Kumar,[phone]
M Kumar,[phone]
P Kumar,[phone]
M Kumar,[phone]
Pradeep Kumar,[phone]
R Kumar,[phone]
V Kumar,[phone]
Rakesh Kumar,[phone]
J Kumar,[phone]
A Kumar,[phone]
A Kumar,[phone]
Parapurath S Kumar,[phone]
N Kumar,[phone]
Praveen Kumar,[phone]
V Kumar,[phone]
A Kumar,[phone]
R Kumar,[phone]
Praveen Kumar,[phone]
J Kumar,[phone]
Praveen Kumar,[phone]
S Kumar,[phone]
Ragesh Kumar,[phone]
Goutham Kumar,[phone]
V Kumar,[phone]
Dharmana Kumar,[phone]
K Kumar,[phone]
J Kumar,[phone]
Praveen Kumar,[phone]
D Kumar,[phone]
Parmod Kumar,[phone]
S Kumar,[phone]
Suby Kumar,[phone]
A Kumar,[phone]
A Kumar,[phone]
Surinder Kumar,[phone]
Rajesh Kumar,[phone]
Selva Kumar,[phone]
Santhosh Kumar,[phone]
Praveen Kumar,[phone]
K Kumar,[phone]
J Kumar,[phone]
C Kumar,[phone]
k kumar,[phone]
A Kumar,[phone]
Praveen Kumar,[phone]
J Kumar,[phone]
Ashok N Kumar,[phone]
V Chandra Kumar,[phone]
D Kumar,[phone]
S Kumar,[phone]
Maria Kumar,[phone]
S KUMAR,[phone]
P Kumar,[phone]
SURESH KUMAR,[phone]
S Kumar,[phone]
Santosh Kumar,[phone]
p kumar,[phone]
V R Kumar,[phone]
Satish Kumar,[phone]
J Kumar,[phone]
Sam Kumar,[phone]
R Kumar,[phone]
M Kumar,[phone]
Nita Kumar,[phone]
Rajender Kumar,[phone]
R Kumar,[phone]
V KUMAR,[phone]
Raman Kumar,[phone]
B Kumar,[phone]
S Kumar,[phone]
S Kumar,[phone]
S Kumar,[phone]
D Kumar,[phone]
R Kumar,[phone]
Pj Kumar,[phone]
Rai Bal Kumar,[phone]
S Kumar,[phone]
S Kumar,[phone]
V KUMAR,[phone]
R Kumar,[phone]
R Kumar,[phone]
R Kumar,[phone]
K Kumar,[phone]
P Kumar Rudra,[phone]
Rashmi Kumar,[phone]
Anil Kumar,[phone]
Shiv Kumar,[phone]
Aprul Kumar,[phone]
A Kumar,[phone]
P Kumar,[phone]
T Melburn-Kumar,[phone]
V Kumar,[phone]
Poonam Kumar,[phone]
C Kumar,[phone]
V Kumar,[phone]
M KUMAR,[phone]
R Kumar,[phone]
A Kumar,[phone]
M Kumar,[phone]
S Kumar,[phone]
P & R Kumar,[phone]
L Kumar,[phone]
R Kumar,[phone]
R KUMAR,[phone]
K Kumar,[phone]
J Kumar,[phone]
S Kumar,[phone]
D Kumar,[phone]
Jagjit Kumar,[phone]
Vimala Krishna Kumar,[phone]
R Kumar,[phone]
A Kumar,[phone]
Santosh Kumar,[phone]
Vijay Kumar,[phone]
Dr Vinod Dr Uma Kumar,[phone]
S Kumar,[phone]
L KUMAR,[phone]
Mifan Kumar,[phone]
S Kumar,[phone]
Padin Kumar,[phone]
Shiue Kumar,[phone]
M Kumar,[phone]
P Kumar,[phone]
A Kumar,[phone]
R Kumar,[phone]
R Kumar,[phone]
E Kumar,[phone]
S Kumar,[phone]
V Kumar,[phone]
M Kumar,[phone]
D Kumar,[phone]
S Kumar,[phone]
Raj Kumar,[phone]
A Kumar,[phone]
S Kumar,[phone]
S Kumar,[phone]
V Kumar,[phone]
P KUMAR,[phone]
A Kumar,[phone]
P Kumar,[phone]
Hope it was helpful.