Request the same webpage as seen in webbrowser - python-3.x

Question

I'm trying to scrape some webpage, but I'm encountering with the problem that the page content is different from what I'm seeing in Firefox
This is my code:
import requests
from bs4 import BeautifulSoup
url = "https://www.sareb.es/es_ES/inmuebles"
with requests.get(url, verify = False) as html_file:
soup = BeautifulSoup(html_file.content, "html.parser")
soup.find_all("h3")
I want to scrape the prices, which are in h3 tags, but the output is not showing them with soup.find_all("h3").
Is there any way to retrieve the "same" webpage?
Thanks

Answer 1

You can use requests for json response, create a loop with the page number to get more results. Total page is 1217.
import requests
url = "https://www.sareb.es/dynamic/assets/json?"
params = {
"lang": "en_US",
"page": "1",
"orderField": "score",
"orderDirection": "DESC",
"compId": "7aa1f42482964610VgnVCMServera5ecbf0aRCRD",
"rtbPage": "Home > Inmuebles",
"compId": "7aa1f42482964610VgnVCMServera5ecbf0aRCRD"
}
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36",
"X-CSRF-TOKEN": "5618a1c8-9d8e-4a88-95a8-2eef4c2b3455",
"X-Requested-With": "XMLHttpRequest"
}
r = requests.get(url, params=params, headers=headers, verify=False)
d = r.json()
results = (d['result']['assetsPage']['content'])
for result in results:
print(result['type'], result['price'], result['city'], result['district'])
Results:
Country House 281.000 € Sueca Valencia/València
Country House To consult Moaña Pontevedra
Country House 152.000 € Corcos Valladolid
Country House 36.000 € Alcalà de Xivert Castellón/Castelló
Office 130.800 € Sagunto/Sagunt Valencia/València
Office 643.370 € Valencia Valencia/València
Office 646.495 € Valencia Valencia/València
Office 3.100 € Valencia Valencia/València
Office 144.700 € / 1.070 € Palmas de Gran Canaria (Las) Palmas, Las
Office 326.400 € / 1.635 € Murcia Murcia
Offices From 60.000 € Colmenar Viejo Madrid
Office 519.300 € / 2.564 € Alicante/Alacant Alicante/Alacant
Office To consult Palma de Mallorca Balears, Illes
Office 97.000 € Santa Lucía de Tirajana Palmas, Las
Offices From 444.200 € Sevilla Sevilla
Offices From 34.700 € Villamayor Salamanca